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The  1981  Image  Generation/Display  Conference  II  is  the  only  conference 
devoted  entirely  to  issues  relevant  to  the  development  and  use  of  imagery 
generated  and  displayed  for  visual  flight  simulation.  The  first  Conference  was 
held  at  Williams  Air  Force  Base,  Arizona,  in  May  of  1977.  The  purpose  of  the 
Conference  is  to  provide  a  forum  for  presenting  and  discussing  topics  concerned 
with  the  imagery  generated  for  out  of  the  cockpit  and  sensor  visual  flight 
simulation.  The  Air  Force  recognizes  that  the  rapid  technological  advances  and 
uses  of  real-time  visual  simulators  requires  that  the  interchange  of  informa¬ 
tion  among  the  user  organizations  be  expanded  in  order  to  promote  new  develop¬ 
ments,  applications,  and  techniques  and  avoid  unnecessary  duplication  of 
efforts. 

Papers  are  solicited  in  all  areas  compatible  with  the  theme  of  the  Con¬ 
ference;  including  engineering  research  and  development,  behavioral  research, 
application  and  techniques,  specific  program's  progress  and  status,  as  well  as 
technical  problems  and  potential  solutions.  Pertinent  topics  include  but  are 
not  necessarily  limited  to: 

1.  Software/hardware  developments,  directly  resulting  in  an  enhancement  of 
image  capabilities. 

2.  Psychological  determination  of  visual  cue  requirements. 

3.  Environmental  data  base  design  and  structure. 

The  Image  Generation/Display  Conference  was  conceived  in  1977  at  the  Opera¬ 
tions  Training  Division  (formerly  the  Flying  Training  Division)  of  the  Air 
Force  Human  Resources  Laboratory  at  Williams  Air  Force  Base,  Arizona.  Since 
its  inception,  the  Conference  has  been  sponsored  by  this  Laboratory  in  a  con¬ 
tinuing  effort  to  pioneer  new  methods  and  devices  for  aircrew  training. 
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INTRODUCTION 
to  the 

1981  Image  Generation/Display  Conference  II 


Eric  6.  Monroe 
Conference  Chairman 


It  is  indeed  a  personal  pleasure  to  welcome  you  to  the  second  image  Generation/ 
Display  Conference.  The  participation  in  the  first  conference  held  in  May  of 
1977  exceeded  my  original  conception  of  a  relatively  small  symposium,  con¬ 
sisting  primarily  of  project  scientists  and  engineers  interacting  at  a  task 
level  basis.  A  total  of  21  behavioral  and  engineering  papers  were  presented 
at  that  conference.  This  year,  31  papers  will  be  presented  and,  upon 
examination  of  the  Proceedings,  I  believe  you  will  find  the  quality  of  the 
papers  and  expertise  of  their  authors  second  to  none  in  their  field.  In 
addition  to  the  papers,  a  panel  discussion  entitled  "What  You  Always  Wanted  To 
Know  About  CIG  But  Were  Afraid  To  Ask"  and  a  debate  with  the  topic  "Is  the  Eye 
Sufficient  To  See"  have  been  established  in  order  to  encourage  more  audience 
participation.  As  you  look  around  and  observe  your  fellow  conferees ,  I  believe 
you  will  find  them  to  be  neither  lacking  in  experience  or  credentials.  In 
attendance  this  year  are  representatives  of  industry,  military,  aerospace,  and 
governmental  and  educational  institutions  from  all  over  the  United  States, 
Great  Britain,  Germany,  France,  and  Canada.  The  result  of  bringing  together 
this  broad  spectrum  of  intellectual  talents  focusing  on  the  central  theme  of 
visual  flight  simulation  should  have  a  synergistic  effect  in  achieving  our 
goals.  The  primary  objective  of  the  Conference  is  to  provide  a  forum  for  the 
exchange  of  information  concerning  engineering  technology  developments  and 
behavioral  research  issues  relative  to  effective  and  efficient  pilot  training 
with  visual  flight  simulators.  The  objective  of  the  Conference  has  not  changed 
over  the  last  four  years,  but  many  of  the  issues  and  technological  capabilities 
have.  Some  of  these  will  be  raised  in  the  papers,  panel  discussion,  and  debate 
over  the  next  few  days.  Others  will  hopefully  arise  out  of  your  personal 
interactions  at  some  of  the  social  functions.  I  believe  you  will  find  the 
environment  at  the  beautiful  Registry  Resort  here  in  Scottsdale,  Arizona,  pro¬ 
vides  an  atmosphere  conducive  to  stimulating  thought  and  communication.  May 
you  find  your  attendence  here  most  pleasant  and  productive. 
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OPENING  REMARKS 


Colonel  Richard  C.  Needham 
Chief,  Operations  Training  Division 
Air  Force  Human  Resources  Laboratory 
Williams  Air  Force  Base,  Arizona 


Colonel  Richard  C.  Needham  assumed  his  duties  as  Operations  Training 
Division  Chief  in  June  1978,  after  having  served  as  Chief,  Flying  Training 
Branch,  Deputy  Chief  of  Staff  for  Personnel,  Headquarters  USAF,  the 
Pentagon.  In  his  current  capacity.  Colonel  Needham  is  responsible  for  the 
operation  and  conduct  of  research  on  the  world's  most  advanced  aircrew 
training  research  device.  A  native  of  Phoenix,  Arizona,  he  was  commissioned 
a  Second  Lieutenant  in  1955,  receiving  his  initial  Air  Force  assignment  as  a 
Radar  Observer  in  the  75th  Fighter  Squadron  at  Presque  Isle  AFB,  Maine.  He 
attended  pilot  training  at  Moultrie  AFB,  Georgia  and  Webb  AFB,  Texas.  After 
attending  gunnery  school  in  F-86F  aircraft,  he  was  assigned  to  Greenville 
AFB  in  1959  as  an  instructor  pilot  in  T-33  aircraft.  He  also  served  for 
five  years  as  a  T-37  instructor  pilot  at  Williams  AFB.  He  received  his 
Bachelor  of  Science  Degree  in  Political  Science  from  the  University  of 
Nebraska  in  1965.  He  is  a  command  pilot  with  over  4500  flying  hours. 
Military  decorations  include  Silver  Star,  Distinguished  Flying  Cross,  Air 
Medal  with  seven  oak  leaf  clusters,  three  Presidential  Unit  Citations  and 
numerous  other  awards  and  decorations. 
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COMMANDER  *  S  COMMENTS 


Colonel  Ronald  W.  Terry 

Commander,  Air  Force  Human  Resources  Laboratory 
Brooks  Air  Force  Base,  Texas 


r  i 


Colonel  Ronald  W.  Terry  is  Commander,  Air  Force  Human  Resources  Labora¬ 
tory,  Brooks  AFB,  TX.  Colonel  Terry  enlisted  in  the  Air  Force  in  June 
1952.  He  graduated  from  Aviation  Cadets  and  was  commissioned  May  1954.  The 
next  5  years  were  spent  in  operational  assignments  with  the  Tactical  Air 
Command  in  F-86H  and  F-100  aircraft  and  a  tour  in  Iceland  with  the  Military 
Airlift  Command. 

After  graduation  from  the  School  of  Business  at  Indiana  University  in 
1961,  Colonel  Terry  was  assigned  to  the  Air  Force  Systems  Command.  He  has 
served  in  a  wide  variety  of  assignments,  including  Flight  Test  as  project 
pilot  and  test  director,  Aide  to  the  Commander,  ASD,  HQ  AFSC  Monitor  for  the 
Air  Force  Project  Shedlight  (Night  Attack  Capability  for  the  Air  Force)  and 
also  assignments  in  the  DCS/Personnel  at  HQ  AFSC.  He  was  previously 
assigned  Assistant  Deputy  Chief  of  Staff /DCS  Procurement  and  Manufacturing, 
Headquarters  Air  Force  Systems  Command. 

Colonel  Terry  is  best  known  as  the  originator  and  Director  of  the  Air 
Force  AC-47  and  AC-130  Gunship  programs.  Not  only  were  these  systems 
designed  and  integrated  by  Air  Force  personnel,  but  these  same  people  saw 
extensive  combat  operations  throughout  US  involvement  in  Vietnam,  Laos  and 
Cambodia.  The  program  office  was  the  focal  point  of  combat  operations  for 
each  model  until  they  were  transitioned  to  the  Tactical  Air  Command.  A  num¬ 
ber  of  program  office  personnel  (including  Colonel  Terry)  are  on  the  list  of 
most  missions  and  combat  hours  flown  in  SEA. 

Colonel  Terry  was  a  distinguished  graduate  of  the  Aviation  Cadets  in 
1954.  He  graduated  with  honors  in  (AFIT)  Management  at  Indiana  University 
in  1961.  He  was  a  distinguished  graduate  of  ICAF  in  1974.  He  was  the  first 
recepient  of  the  USAF  Harold  Brown  Award.  He  also  received  the  Harvey  C. 
Knowles  Award  from  the  American  Ordnance  Association  and  the  Citation  of 
Honor  Award  from  the  Air  Force  Association.  From  the  Air  Force  he  has  been 
presented  the  Legion  of  Merit  with  three  oak  leaf  clusters,  the 
Distinguished  Flying  Cross  with  two  oak  leaf  clusters.  Air  Medal  with  eight 
oak  leaf  clusters,  the  Bronze  Star,  and  other  decorations  and  awards. 
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Chairman 

Dr.  Bernard  A.  Kulp 

Chief  Scientist,  Director  of  Laboratories 
Headquarters,  Air  Force  System  Command 
Andrews  Air  Force  Base,  D.C. 


Dr.  Bernard  A.  Kulp  Is  a  native  of  Ohio  and  received  his  Ph.D.  In  Phy¬ 
sics  from  The  Ohio  State  University.  He  spent  eleven  years  In  the  Aerospace 
Research  Labs  of  the  Office  of  Aerospace  Research  at  Wright-Patterson  Air 
Force  Base,  Ohio,  where  he  conducted  research  in  radiation  damage  in  semi¬ 
conductors.  He  is  the  author  of  20  articles  in  Physical  Review.  Journal  of 
Applied  Physics,  and  other  scientific  journals,  and  was  Adjunct  Professor  of 
Physics  at  the  Air  Force  Institute  of  Technology.  From  1965  to  1969  he  was 
engaged  in  design,  development  and  testing  of  electro-optic  devices  and  sys¬ 
tems  for  night  reconnaissance  and  guidance  and  control.  He  joined  Air  Force 
Armament  Lab  in  1969  as  Chief  Scientist  to  assist  in  the  development  of  a 
Missile  Systems  Technology  capability.  He  transferred  to  Air  Force  Systems 
Command  in  1975  to  be  Chief  Scientist  to  the  Director  of  Laboratories. 

He  was  awarded  the  Exceptional  Civilian  Service  Medal  for  work  in 
electro-optics. 

He  is  a  member  of  many  professional  and  scientific  societies  and  is  a 
Fellow  of  the  American  Association  for  the  Advancement  of  Science. 
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COMPUTATIONAL  ANALYSIS:  A  TECHNIQUE  FOR 
IMPROVING  THE  VISUAL  SIMULATION  OF  TERRAIN: 
(as  applied  to  low  level  (light) 


Kent  A.  Stevens 


Kent  A.  Stevens  obtained  his  Ph.D.  from 
the  Massachusetts  Institute  of  Technology 
in  Artificial  Intelligence,  and  his  M.S.  and 
U  S.  arc  from  the  University  of  California. 
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COMPUTATIONAL  ANALYSIS:  A  TECHNIQUE  FOR 
IMPROVING  THE  VISUAL  SIMULATION  OF  TERRAIN: 

(as  applied  lo  low  level  flight)1 

Abstract 

Images  carry  information,  and  for  fligiu  simulation  one  needs  to  understand  how  that  information  is 
conveyed  in  the  image,  relative  to  the  human  visual  processes  dial  utilize  the  information.  Developing  a 
useful  definition  of  "information  content",  however,  has  been  a  frustrating  enterprise.  While  images  carry 
information,  it  also  true  that  images  arc  ambiguous,  and  that  additional  information  is  necessary  for  their 
interpretation.  Human  vision  interprets  the  image,  in  part,  by  utilizing  properties  of  die  natural  world  (such 
as  those  which  Gibson  [F)50|  brought  to  our  attention)  which  constrain  the  possible  interpretations.  Those 
constaints  are  amenable  to  precise  mathematical  description,  and  dieir  theoretical  study  within  a  larger 
computational  mework  (which  we  describe  here)  yeilds  useful  insights  into  "information  content"  and  thus 
how  visible  surfaces  might  be  better  depicted  in  the  simulator. 


I.  Introduction 

l'hc  Air  Force  has  asked  the  Committee  on  Vision  of  the  National  Research  Council  to  recommend  research 
strategics  for  improving  the  visual  simulation  of  low  level  flight  (I.I.F).  Fmphasis  is  placed  on  I.I.F  because  it 
is  a  flight  regime  of  increasing  importance  in  combat,  and  also  a  regime  where  significant  difficulty  has  been 
experienced  in  its  simulation.  Pilots  who  arc  expert  in  actual  I.I.F  often  cannot  duplicate  dieir  performance 
in  die  simulator.  When  "on  die  deck"  in  die  simulator  they  have  difficulty  both  in  following  die  contour  of 
the  terrain  at  low  altitude  (without  crashing)  and  in  judging  altitude.  In  part,  die  difficulty  is  technological, 
for  instance  the  very  rapid  motion  of  terrain  detail  across  the  display  requires  fast  display  processors.  Other 
difficulties  arc  perceptual:  the  visual  displays  must  convey  an  adequate  3-D  understanding  of  die  terrain  over 
which  the  pilot  must  fly.  The  combination  of  technological  and  perceptual  problems  has  prompted  some 
re-examination  of  the  methods  for  terrain  depiction.  This  article  describes  a  new  technique  for  improving  the 
v  isual  simulation  of  terrain,  as  applied  to  the  simulation  of  LLF. 

The  introduction  will  discuss  dircc  background  issues.  The  first  concerns  flight  training.  Research  and 
development  efforts  regarding  simulators  for  flight  training  should  be  evaluated  in  terms  of  dieir  impact  on 
training.  However,  I.I.F  simulation  presents  some  basic  visual  problems  that  must  be  solved  regardless  of 
whether  the  simulator  is  used  for  training  or  any  other  purpose.  Hence  we  concentrate  here  solely  on  the 
visual  problems  of  1.1  F  simulation.  Next,  we  show  that  although  C1G  displays  caricature  the  real  world,  they 
can  still  convey  useful  3-D  information.  With  that  observation  we  turn  to  problems  of  defining  "information 
content".  Finally,  we  discuss  a  computational  methodology  for  studying  human  vision  that  approaches 
complex  systems  at  several  levels  of  detail  or  specificity,  where  different  theoretical  tools  arc  appropriate  at 
each  level.  Iliis  approach  has  lead  to  succinct  and  precise  descriptions  of  various  processes  of  human  vision 
(such  as  stercopsis  and  edge  detection,  see  later).  That  point  is  crucial  for  its  application  to  flight  simulation: 
even  diough  we  currently  have  only  a  limited  understanding  of  the  overall  visual  system,  many  facts  that  we 
do  know  translate  into  important  suggestions  for  simulator  scene  generation.  The  rest  of  the  article  attempts 
to  show  how  that  insight  comes  about 


1.  This  article  will  appear  as  an  appendix  to  the  forthcoming  report  by  the  Committee  on  Vision  of  the  National  Research  CouncU  on 
research  strategics  for  the  visual  simulation  of  low  level  flight 
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1.1  Some  remarks  regarding  training 


The  flight  simulator  that  we  have  in  mind  is  a  device  for  training  low  level  flight.  But  the  fact  that  simulated 
1.1. F  is  difficult  to  fly  for  an  experienced  pilot,  let  alone  a  trainee,  places  the  following  training  issues  in  a 
different  perspective. 

It  is  often  said  that  a  training  device  should  be  optimized  for  the  particular  training  task  it  will  be  used  for. 

Thus,  for  instance,  die  distinction  between  training  a  skill  to  a  nov  ice  and  training  for  performance  or  skill 
maintainancc  should  be  reflected  by  the  simulator  system.  But  it  is  doubtful  that  the  suite  of  die  art  of  I.LF 
simulation  is  sufficiently  advanced  to  make  this  distinction  worthwhile.  We  need  a  visual  simulation  diat  is 
adequate  for  die  experienced  pilot  before  we  may  consider  optimizing  for  particular  sorts  of  training. 

But  one  may  argue  that  in  emphasizing  the  visual  simulation  without  regard  for  its  role  as  a  device  for 
training  pilots,  one  might  inadvertently  concentrate  on  visual  problems  diat  arc  unimportant  to  training,  or 
overlook  problems  that  are  important  to  training.  In  diosc  training  Lisks  for  which  the  visual  simulation  is  at 
least  adequate  (as  in  landings,  inflight  refueling,  and  formation  flying),  die  simulation  should  be  optimized 
for  training  diat  task.  But  for  I.I.K  the  visual  display  is  still  inadequate,  hence  one  needs  to  concentrate  on 
impro>  ing  it  before  addressing  issues  of  training,  llic  pragmatic  view  uiken  here  is  that  there  arc  two  distinct 
problems  facing  simulation:  making  the  display  adequate  in  basic  perceptual  terms,  and  making  it  particularly 
useful  for  training.  Kffort  on  one  problem  should  not  be  made  at  the  cxlusion  of  the  other.  However,  we  will 
not  not  deal  with  issues  of  training  here. 

1.2  Simulation  is  often  caricature 

When  the  CIG  simulation  of  l.l. I'  is  compared  to  the  real  world  seen  from  a  low  flying  aircraft,  the  simulation 
seems  oversimplified  and  somewhat  corrupted  by  artifacts,  such  as  the  stuttering  effect  that  results  when  the 
CIG  system  cannot  update  die  display  in  the  periphery  sufficiently  often  given  die  large  angular  velocities 
associated  with  I.I.K.  One  might  conclude  that  the  3-1)  impression  would  be  adequate  if  only  one  had  greater 
display  capability.  But  it  is  conceivable  diat  even  a  hundred- fold  improvement  would  not  solve  the  problem. 

Insufficiency  of  detail  is  not  the  whole  story.  A  simulation  scene  is  not  merely  a  simplification  of  a  real 
scene;  there  arc  dramatic  and  qualitative  differences.  In  reducing  the  bandwidth  of  a  real  visual  scene  to 
manageable  proportions,  there  is  no  straightforward  sampling  technique  available  -  we  cannot  merely  take 
every  «-th  bit  of  information,  as  it  were.  Instead,  we  construct  a  novel  and  unique  world  out  of  a  relatively 
few  edges,  and  perhaps  shading,  color,  points,  and  lines. 

The  visual  world  we  construct  in  the  simulator  shares  many  similarities  with  the  real  world.  The  single 
most  important  similarity  is  that  the  rules  of  perspective  geometry  are  preserved  in  the  simulation.  That 
means  the  visual  world  in  the  simulation  behaves  optically  as  we  expect.  This  point  should  not  be  underrated, 
for  it  is  responsible  for  much  of  the  success  of  visual  simulation.  Specifically,  the  perspective  transformation 
by  which  the  simulator  CIG  system  projects  3-1)  points,  lines,  and  surfaces  from  the  terrain  model  onto  the 
2-1)  display  screen  (and  in  turn,  onto  the  retina  so  long  as  the  display  is  viewed  from  the  proper  location) 
precisely  corresponds  to  the  way  real  3-D  points  project  onto  the  retina.1  Consequently,  any  static  view  is 
reproduced  with  geometric  precision  (e.g.  the  proper  "texture  gradient"  is  generated  because  increasingly 
distant  surface  elements  appear  progressively  smaller).  Also,  if  the  surfaces  are  modelled  as  opaque,  the  CIG 
system  (usually)  has  the  capability  to  make  nearer  surfaces  occlude  from  view  those  that  are  farther. 

Moreover,  if  we  move  in  space,  the  continuous  changes  in  perspective  of  points  and  surfaces  also  corresponds 
geometrically  to  what  happens  in  the  real  world.  Consequently  we  also  have  geometrically  correct  "optic 
flow",  "motion  parallax",  and  so  forth  (see  later).  It  is  the  fact  that  perspective  projection  is  accurately 
duplicated  by  the  CIG  system  that  was  responsible  for  the  compelling  effect  of  space  in  early  night  flight 
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1.  But  see  [Kraft,  tt  aL  1980]  for  engineering  considerations  • 


simulators,  where  only  moving  luminous  dots  (such  as  runway  lights  and  city  lights)  were  projected.  As 
opaque  surface  shading  was  added  and  daytime  scenes  were  simulated,  die  additional  gains  came,  in  pan, 
from  the  visually  apparent  occlusion  of  distant  objects  by  nearer  ones.  But  along  with  attempts  to  simulate 
daytime  scenes  came  the  realization  that  tire  simulation  is  very  different  from  the  real  thing. 

One  difference  of  probably  minor  importance  is  that  the  display  is  effectively  a  monocular  presentation 
(see  later).  But  just  to  note  a  few  of  the  profound  differences  between  the  real  scene  and  the  simulated 
counterpart,  observe  that  real  surfaces  have  detail  at  all  scales,  and  that  new  detail  is  continuously  revealed  as 
one  approaches  a  surface.  But  the  limited  detail  that  most  CIG  systems  can  display,  combined  with  their 
resolution  limits,  result  in  drastically  simplificated  scenes.  (This  fact  may  be  particularly  relevant  to  the  LLF 
question,  as  pilots  in  real  flight  conceivably  use  fine  recognizable  detail,  of  vegetation  and  rocks,  in  order  to 
judge  their  distance  from  approaching  terrain.)  Another  difference  is  that  real  surfaces  arc  not  restricted  in 
their  3-D  shape  (consider  a  rolling  desert  floor,  or  an  eroded  canyon  wall).  But  the  usual  computer 
representation  of  curved  surfaces  in  terms  of  planes  bounded  by  polygons  results  in  a  dramatically  different 
sort  of  terrain  than  one  that  would  be  found  in  nature.  It  is  not  simply  the  fact  that  the  surface  is  piecewise 
planar  rather  than  continuously  curved,  it  is  also  not  true  that  sharp  straight  boundaries  occur  at  every 
junction.  Those  straight  boundaries  arc  also  used  to  denote  field  boundaries  (such  as  the  checkerboards 
which  resemble  agricultural  land).  In  fact,  the  basic  element  in  simulation  for  representing  surfaces  is  an  edge 
across  which  screen  intensity  is  sharply  discontinuous.  That  edge  may  represent  a  sharp  physical  feature  such 
as  a  cliff,  ridge,  or  corner  of  a  building,  or  a  place  where  the  surface  reflectance  sharply  changes  such  as  the 
boarder  of  a  runway  or  the  edge  of  a  checkerboard  square.  Some  display  systems  have  shading  and 
modulation  of  contrast  to  approximate  aunosphcric  haze  or  fog.  but  generally  the  CIG  edges  arc  equally 
sharp  and  homogeneous  --  qualities  that  are  not  natural.  Another  point  is  that  actual  surfaces  have  physical 
texture  in  relief  above  the  mean  surface  level  (bushes  above  the  ground,  peaks  and  troughs  of  waves)  but  this 
is  costly  to  generate  in  current  simulator  systems.  One  final  difference  should  be  pointed  out:  actual  surfaces 
reflect  light  in  a  complex  way  depending  upon  the  orientation  of  the  light  source  (the  sun  and  the  overall  sky 
illumination),  the  orientation  of  the  viewer  relative  to  the  surface,  and  the  physical  properties  of  the  surface. 
But  generally  the  intensities  of  simulated  surfaces  arc  unnaturally  constant  and  homogeneous  --  even  the  most 
sophisticated  techniques  for  shading  surfaces  arc  highly  simplified. 

'Ihis  discussion  is  not  intended  to  point  out  the  already-known  fact  that  simulator  scenes  arc  unrealistic.  Its 
purpose  is  to  show  that  a  CIG  display  is  not  merely  a  simplification  of  the  real  world  --  it  is  inevitably  a 
caricature.  Should  we  regard  the  caricature-like  qualities  of  CIG  display  as  detrimental?  Not  necessarily,  for 
human  vision  has  a  remarkable  ability  to  ignore  simplifications  in  illustrations.  For  instance,  we  are  all 
familiar  with  the  strong  visual  impact  that  even  simple  line  drawings  provide,  such  as  seen  in  engineering  and 
mathematics  texts,  and  in  assembly  instructions.  While  they  are  highly  unrealistic,  we  seldom  attend  to  that 
fact.  Since  they  carry  the  necessary  3-D  information,  they  serve  their  purpose:  it  docs  not  matter  that  actual 
surfaces  give  rise  to  more  complicated  images.  Similarly,  it  may  be  argued  that  simulation  displays  of  terrain 
may  be  adequate  (for  purposes  of  training,  etc.)  and  yet  highly  unrealistic  and  caricatured. 

1.3  Problems  with  intuitive  and  geometric  argument 

Thus  we  have  that  the  CIG  display  may  carry  the  necessary  3-D  information  despite  the  unrealistic  qualities 
just  described.  But  we  still  have  to  formalize  what  we  mean  by  "necessary  3-D  information".  In  that  pursuit 
it  should  be  stressed  that  our  intuition  should  not  be  trusted,  intuition  is  often  wrong  regarding  the  way  the 
human  visual  system  operates.  In  short,  what  we  naively  believe  to  govern  our  visual  perceptions  is  often  not 
the  case.  It  is  worthwhile  examining  one  such  example  in  detail.  The  example  involves  the  familiar  "texture 
gradient",  specifically,  that  the  texture  density  seems  intuitively  to  be  the  crucial  depth  cue  in  texture 
gradients  the  higher  the  density,  the  greater  the  distance  to  the  surface  [Gibson  1950]  (and  see  the  geometric 
analysis  in  [Purdy  I960}),  in  fact,  distance  cannot  be  inferred  directly  from  density  (the  following  argument  it 
summarized  from  (Stevens  1980]).  The  reason  is  that  texture  density  is  a  function  not  only  of  the  distance  to 


8 


the  surface  but  also  of  the  slant  of  the  surface  relative  to  the  viewer  (the  greater  the  slant,  the  greater  the 
foreshortening  and  hence  the  greater  the  texture  density).  In  the  case  of  an  arbitrary  surface  one  cannot 
decouple  the  relative  contributions  to  the  density  gradient  caused  by  foreshortening  from  that  due  to  distance. 
Consequently  one  cannot  infer  distance  from  texture  density.  This  argument  provides  an  explanation  for  the 
largely  ignored  psychophysical  evidence  (eg..  [Smith  &  Smith  1957;  Braunstein  1968;  Braunstcin  &  Payne 
1969])  that  texture  density  is  an  ineffective  cue  to  distance.  This  result  is  contrary  to  our  intuitions.  It  points 
out  that  we  must  go  beyond  introspection. 

In  the  same  vein,  it  is  not  sufficient  merely  to  examine  the  geometric  properties  of  perspective  projection. 
For  instance,  the  analysis  of  texture  gradients  that  Purdy  [I960]  performed  set  out  to  find  geometric  basis  for 
the  hypotheses  that  Gibson  [1950]  set  forth  (such  as  the  one  just  discussed).  But  such  geometric  analyses  must 
be  carefully  regarded,  as  they  implicitly  embody  certain  geometric  assumptions,  such  as  that  the  ground  is 
globally  planar  (see  [Purdy  I960]).  I'hc  mathematical  relations  derived  with  those  assumptions,  of  course,  do 
not  hold  when  the  physical  surfaces  arc  not  so  constrained  (e.g..  when  the  textured  surface  is  not  planar).  It  is 
not  a  straightforward  matter  to  set  down  the  geometric  relations  that  provide  the  basis  for  visual  interpretation 
of  the  image.  The  relations  necessarily  incorporate  constraints,  and  it  is  a  separate  and  non  trivial  matter  to 
verify  that  those  constraints  arc  adopted  by  the  actual  perceptual  processes  of  human  vision.  (For  further 
discussion  on  the  matter  of  constraints  and  its  relation  to  Gibson's  "direct  perception"  hypothesis,  the  reader 
is  referred  to  [Ullman  1980! ) 

1.4  How  is  information  encoded  in  the  image? 

Image  information  coru'MM  is  usually  discussed  in  terms  of  depth  cues  such  as  texture  gradients,  optic  flow, 
and  stcrcopsis.  But  to  :v  ireful,  in  particular  to  be  useful  for  flight  simulation,  those  terms  must  be  defined 
more  precisely  thtur  is  usual  in  the  psychological  literature  (see  also  [Hennesy  cl  al.  1980;  Semple  et  al.  1980]). 
To  illustrate,  consider  the  difficulty  a  simulator  designer  would  have  trying  to  apply  a  general  fact  about  optic 
flow.  For  argument  suppose  he  reads  that  optic  flow  provides  information  about  the  direction  of  travel 
because  of  the  streaming  of  detail  radially  away  from  the  point  he  is  approaching  [Gibson  I950J.  If  the 
designer  learns  that  pilots  often  lose  visual  orientation  in  the  simulator  (while  maneuvering,  say),  the  designer 
would  recall  that  optic  flow  is  relevant.  But  the  designer  is  now  stuck,  in  a  word,  because  optic  flow  was  not 
defined  with  sufficient  specificity  that  the  designer  might  translate  that  suggestion  into  an  improved  visual 
display.  It  is  a  rather  subtle  point  that  optic  flow  is  inevitable  whenever  one  moves  through  other  than  empty 
space.  Optic  flow  is  a  consequence  of  perspective  projection  and  motion,  so  regardless  of  how  the  terrain  is 
depicted,  if  one  moves  across  it.  the  resulting  movement  of  detail  across  die  visual  field  is  optic  flow. 

The  designer  is  faced  with  a  terrain  simulation  that  is  apparently  inadequate,  seeks  to  improve  it,  and 
knowns  that  optic  flow  is  relevant.  So  he  considers  the  variables  that  govern  optic  flow,  those  being  the 
parameters  of  motion  relative  to  the  surface  as  well  as  the  qualities  of  the  surface  texture  itself.  (The  faster 
and  lower  the  aircraft,  the  more  pronounced  the  apparent  optic  flow;  also,  the  more  densely  textured  the 
surface,  the  greater  the  effect)  Of  these  variables,  it  is  only  the  surface  texture  that  the  designer  may 
manipulate,  but  how  to  do  so?  Insight  would  come  from  knowing  something  about  how  the  human  visual 
system  measures  and  internally  encodes  moving  texture  and  how  it  extracts  3-D  information  such  as 
orientation  from  those  measurements.  We  cannot  dismiss  this  issue  in  a  cavalier  manner,  trusting  that  the 
human  visual  system  will  extract  what  it  needs  so  long  as  the  CIG  display  is  rich  enough  in  detail.  Current 
(and  foreseeable)  CIG  display  technology  docs  not  (and  probably  will  not)  have  the  capability  of  reproducing 
the  complex  visual  texture  presented  by  real  LTF  over  natural  terrain.  Rather,  we  should  devise  more 
efficient  uses  of  the  CIG  system’s  ability  to  generate  texture.  We  might  think  of  this  as  "impedance 
matching",  of  matching  the  type  of  texture  that  is  displayed  with  the  type  of  visual  processing  we  make  on 
that  texture,  so  as  to  optimize  the  information  transfer.  For  instance,  the  visual  display  might  concentrate  its 
detail  at  the  (instantaneous)  point  of  gaze  and  capitalize  on  the  fact  that  visual  resolution  degrades  with 
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eccentricity.1  But  much  more  thought  is  needed;  it  is  not  enough  to  discuss  what  might  be  omitted  (such  as 
detail  in  the  periphery)  but  what  might  be  included  in  the  texture.  Such  thought  requires  that  one  examine 
the  visual  processes  with  more  analytically  than  in  simple  "depth  cue"  terms. 

1.5  A  computational  information  processing  approach 

We  now  sketch  an  approach  to  die  study  of  vision  w  hich  has  application  to  visual  simulation.  It  is  probably 
well  to  remark  at  die  outset  that  this  approach  stands  among  many  approaches  to  vision  research.  Later  we 
will  see  how  this  approach,  because  of  its  different  perspective,  emphasizes  certain  issues  of  terrain  depiction 
that  have  not  been  addressed  previously. 

It  has  long  been  regarded  that  vision  embodies  processes  that  derive  information  about  the  visual  world 
from  the  interpretation  of  retinal  images.  To  be  sure,  virtually  every  phrase  in  the  previous  statement  has 
been  contested,  cast  in  a  different  light,  or  relatively  emphasized  by  a  particular  theoretical  viewpoint.  For 
instance,  Gibson  [1950,  1979)  characterizes  human  vision  as  direct  registration  of  higher  order  variables  in  the 
visual  stimuli,  thereby  downplaying  the  need  for  interpretation  and  information  processing  (but  see  [Ullman 
1979]).  There  are  some  conflicting  dicorics  of  visual  processing,  but  largely  the  differences  arc  of  emphasis. 
For  instance,  some  researchers  concentrate  on  the  fact  that  our  perception  is  often  as  much  a  product  of  what 
we  expect  as  what  is  objectively  present  in  die  image  (c.g.  [Rock  1977;  llochbcrg  1978]).  Others  concentrate 
on  the  fact  that  natural  scenes  have  richly  redundant  and  consistent  sources  of  information,  and  argue  that 
since  human  vision  probably  capitalizes  on  that  fact,  experiments  using  simplified  stimuli  must  be  carefully 
interpreted  (e.g.  [I  labor  &  Hershenson  1980]). 

Another  theoretical  viewpoint  has  been  developed  largely  by  Marr  [1976.  1978,  1981;  Marr  &  Poggio  1977] 
which  examines  the  information  processing  aspects  of  vision.  The  value  of  this  approach  to  vision  research 
stems  largely  from  recognizing  (i)  dial  human  vision  involves  complex  information  processing,  and  (ii)  that 
complex  systems  arc  feasibly  understood  by  us  only  when  described  at  several  levels  of  detail  and  abstraction. 
Hie  first  point  is  well  accepted  by  psychologists  (in  fact,  some  researchers  find  die  complexity  overwhelming 
and  believe  we  will  never  understand  the  processes  of  vision  because  of  their  complexity).  The  second  point 
is  well  accepted  by  engineers,  and  particularly  computer  scientists,  that  see  the  need  for  making  clean 
distinctions  betw  een  the  purposes  or  goals  of  a  complex  device,  die  mcdiods  by  which  it  achieves  those  goals, 
and  the  particular  nuts-and-bolts  details  of  how  they  arc  carried  out.  The  vocabulary  used  for  describing  what 
a  system  accomplishes  is  very  different  dian  that  used  for  describing  how  it  docs  that  (at  various  levels  of 
detail,  culminating  in  die  physics  of  transistors,  or  whatever  comprise  the  basic  building  blocks  of  the  system). 
Complex  information  processing  systems  require  this  sort  of  multi-level  description;  any  one  level  of 
description,  alone,  would  be  inadequate.  For  instance,  the  circuit  diagram  of  a  digital  computer  at  the  level  of 
individual  transistors  and  capacitors  would  be  incomprehensible,  if  it  were  not  for  additional  descriptions  of 
how  the  electronic  components  implement  logical  functions  such  as  "nand"  and  "or",  and  how  those 
functions  arc  sequenced  to  achieve  processes,  and  so  forth.  Marr  argues  that  biological  information 
processing  should  be  similarly  approached.  Neurophysiology  and  anatomy  provide  us  some  understanding  of 
the  detailed  architecture  of  the  visual  system,  but  just  as  studying  the  physics  of  a  transistor  reveals  nothing 
about  operating  systems  or  Fortran  compilers,  neither  docs  study  of  neural  synapses  reveal  the  principles  of 
stcrcopsis  or  motion  perception. 

This  computational  approach  describes  visual  processes  at  several  levels.2  Ideally  one  like  to  would 


I.  One  should  be  careful  here,  because  while  resolution  degrades  with  eccentricity,  temporal  processing  is  still  strong  (r.g.  flicker  and 
motion  is  readily  detected  in  the  periphery). 

2  See.  e.g.  [Marr  &  Hildreth  19R0;  Richter  &  Ullman  1980)  for  detailed  theories  of  retinal  processing  and  edge  detection.  (Marr  A 
Poggio  1979:  Mayhew  &  I  'risby  1980:  Grimson  1980]  for  computational  theories  of  stcrcopsis.  [Ullman  1979]  for  visual  motion.  |SleveM 
1980, 1981]  for  texture  gradients  and  surface  contours. 
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understand  some  visual  process  (such  as  stcrcopsis)  at  all  levels,  from  die  abstract  and  mathematical  down  to 
die  level  of  neural  implementation.  But  it  is  only  for  certain  very  early  stages  of  visual  processing  (such  as 
retinal  function)  that  neurophysiological  details  arc  feasibly  incorporated  into  a  computational  theory  as  of 
yet  (see  [Man  &  Hildreth  1980:  Richter  &  Ullinan  1980]).  Of  course,  one  can  have  significant  and  useful 
understanding  of  a  complex  mechanism  without  knowing  its  implementation  details.  By  and  large,  the 
theories  emerge  "top-down",  with  die  initial  insights  gained  at  the  level  of  i vital  die  visual  system  computes 
(regarding,  for  instance,  stcrcopsis)  and  die  theoretical  basis  for  that  computation.  This  sort  of  research  is 
reminiscent  of  Gibson  and  his  followers,  for  one  is  concerned  w  ith  geometric  constraints  and  feasiblity.  But, 
in  contrast,  the  computational  approach  concerns  itself  with  die  form  in  which  visual  information  is  encoded 
or  made  explicit  within  the  visual  system.  T  he  3-D  representations  arc  described  abstractly,  not  at  at  the 
neural  level  (that  sort  of  understanding  is  far  from  us  at  the  present)  [Marr  1976,  1978;  Marr  &  Nishihara 
1978:  Stevens  1980]. 

In  formalizing  die  computations  that  underlie  vision,  one  makes  rigorous  the  fonn  in  which  the  visual 
information  is  made  explicit  by  die  visual  system,  the  sources  of  that  information  in  die  image  and  how  that 
information  is  actually  extracted  and  measured,  and  finally,  die  computational  constraints  that  are  necessary 
to  interpret  diat  information.  This  sort  of  vision  research  has  relevance  to  simulation,  therefore,  because 
understanding  how  information  is  extracted  tells  us  how  it  should  be  presented.  Also,  die  constraints  that 
visual  processes  incorporate  are  essentially  perceptual  assumptions  about  the  nature  of  the  visual  world. 
Those  assumptions  arc  cither  always  valid,  because  of  the  physics  of  solid  objects,  perspective,  and  optics,  or 
die  assumptions  arc  usually  valid  because  of  the  statistical  properties  of  the  world  (sec  [Ullman  1979]  for 
discussion  of  assumptions  in  computational  terms,  and  [Gibson  1979]  for  the  related  notion  of  "ecological 
optics").  These  assumptions  have  relevance  to  simulation,  for  in  laying  out  a  simulated  3-D  environment  it  is 
important  to  have  its  properties  match  our  visual  assumptions,  otherwise  our  perceptual  reconstruction  of  that 
environment  will  not  correspond  to  what  was  intended.  A  specific  example  will  be  given  concerning  the 
assumptions  underlying  our  interpretation  of  surface  contours.  Those  results  arc  primarily  geometric;  other 
relevant  contributions  to  simulator  research  pertain  to  efficient  depictions  of  texture. 

With  this  introduction,  the  rest  of  the  article  reflects  on  the  problems  of  depicting  terrain  for  flight 
simulation  from  the  computational  perspective. 

2.  Shape,  orientation,  and  scale 

Let  us  consider  in  general  terms  what  the  pilot’s  visual  system  must  accomplish  in  order  that  an  aircraft  may 
be  maneuvered  at  low  altitude  over  actual  terrain.  Of  course,  the  shape  of  the  terrain  must  be  perceived  in 
3-D,  so  that  a  close  cardi-hugging  course  might  be  followed.  But  not  merely  the  shape  must  be  known,  but 
also  its  size  or  scale.  A  small  hill  and  a  mountain  might  have  the  same  shape  but  differ  significantly  in  scale. 
Wc  will  need  to  become  more  specific  about  shape  and  scale,  but  let  us  continue  informally  for  a  moment 
One  must  also  know  his  orientation  and  attitude  relative  to  the  terrain.  The  pilot,  it  is  reported,  envisions 
himself  as  following  a  path  through  space  above  the  ground,  where  he  keeps  a  mental  trace  of  where  he  has 
been,  where  he  is,  and  a  projection  in  from  of  him  of  where  he  is  going.  In  large  part  this  must  come  from  his 
visually  perceiving  his  orientation  relative  to  the  terrain.  We  must  also  be  specific  about  what  this  entails. 

Wc  have  singled  out  shape,  scale,  and  orientation  as  three  classes  of  visual  information  necessary  for  flying 
over  terrain.  Although  wc  are  being  informal  in  our  use  of  the  terms,  these  three  notions  seem  amenable  to  a 
precise  definition  eventually.  But  is  this  decomposition  of  the  visual  requirements  for  LLF  into  the 
perception  of  shape,  scale,  and  orientation  the  best  decomposition  for  purposes  of  improving  the  visual 
simulation?  It  certainly  seems  that  each  form  of  information  is  basic  and  necessary.  On  the  other  hand, 
concern  is  perhaps  warranted  that  the  shape,  scale,  and  orientation  are  not  sufficiently  inclusive;  we  might  be 
omitting  some  different  quality  of  3-D  information  which  is  necessary  for  LLF.  (Note  that  we  are  not 
considering  information  about  tactical  targets,  navigation  cues,  and  so  forth.  Rather,  we  are  solely  concerned 
with  the  information  that  must  be  gathered  by  the  visual  system  so  that  the  pilot  might  fly  just  above  the 
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terrain.)  We  do  not  know  the  answer  to  that  question;  nonetheless,  the  shapc-oricntation-scalc  approach  to 
describing  necessary  information  should  prove  useful  even  if  not  the  "whole  story". 

'Hie  distinction  between  shape  and  scale  is  important.  Human  vision  is  often  surprisingly  imprecise  about 
scale:  we  appreciate  the  shape  of  a  microorganism  from  a  scanning  electron  micrograph  without  any  concern 
for  (or  appreciation  of)  its  size;  likewise  an  astrophotograph  of  a  nebula  imparts  in  us  no  sense  of  scale.  This 
phenomenon  is  not  difficult  to  explain  theoretically;  it  is  likely  that  the  internal  representations  of  visual 
shape  arc  inherently  scale-independent  (see  the  notion  of  scales  in  [llabcr  &  Hershenson  I980J  and  the 
representations  of  surface  orientation  in  [Attneavc  1972)  and  [Marr  1978]).  Furthermore,  the  visual  system 
probably  treats  as  distinct  perceptual  problems  the  determination  of  an  object’s  shape  and  size.  This 
distinction  between  shape  and  scale  is  often  non-intuitivc  because  in  everyday  scenes  we  usually  have  rich 
sources  to  both  forms  of  information;  there  a^e  many  convergent  perceptual  processes  that  provide 
information  about  shape,  and  there  arc  many  means  by  which  we  directly  and  indirectly  determine  the  actual 
size  and  distance  of  visible  objects.  The  scale  of  our  immediate  surrounds  is  usually  so  precisely  known  that 
we  interact  in  it  with  grace  and  facility.  Rut  we  arc  also  faced  with  natural  situations  in  which  the  scale 
information  is  relatively  impoverished.  One  ease  with  which  most  of  us  arc  familiar  concerns  viewing  the 
earth  from  the  air,  where  the  shape  of  the  mountains  or  canyons  are  clear  to  us.  but  their  size  is  appreciated 
only  if,  say,  we  detect  a  cabin,  or  a  car  on  a  road.  We  then  experience  a  sudden  and  sometimes  shocking 
appreciation  of  the  scale  of  the  scene  and  simultaneously,  of  the  actual  distances  involved.  (The  apparent 
shape  remains  unchanged,  however,  after  we  learn  the  scale,  it  is  as  if  independent  information  is  added  to 
our  perception.)  The  crucial  points  we  should  draw  are  two:  (i)  scale  and  shape  arc  distinct  forms  of 
information  about  the  3-D  world,  and  (ii)  both  forms  of  information  are  needed  in  order  to  interact  precisely 
with  the  world. 

The  third  class  of  3-D  information,  orientation,  has  many  facets  and  is  closely  intertwined  with  shape. 
First  we  will  discuss  what  we  mean  by  orientation,  then  show  how  shape  and  orientation  relate.  Gibson  [1950) 
discusses  two  qualities  of  orientation,  one  local,  the  other  global.  The  local  orientation  of  surfaces  is  defined 
relative  to  the  viewer,  one’s  orientation  in  space  is  defined  globally  relative  to  the  surrounding  visual 
environment.  Let  us  consider  each  in  turn.  It  is  natural  to  visualize  the  orientation  of  a  patch  of  surface 
relative  to  oneself.  The  slant  angle  between  the  line  of  sight  and  the  normal  to  the  surface  is  one  way  of 
quantifying  this.  When  viewing  a  large  planar  surface  such  as  the  ground  seen  from  the  air,  there  is  a  wide 
range  of  relative  surface  orientation  (zero  slant  directly  below  the  aircraft,  slant  approaching  90  degrees 
towards  the  horizon).  But  we  think  of  the  planar  surface  as  having  one  orientation,  not  merely  a  local 
orientation  that  depends  on  which  patch  of  the  surface  we  view.  If  the  plane  rolls,  we  see  the  orientation  of 
the  ground  change  relative  to  us.  We  clearly  have  the  ability  to  judge  orientation  more  globally,  e.g.,  relative 
to  the  horizon. 

We  have  discussed  two  sorts  of  orientation  information:  information  about  the  attitude  of  individual 
patches  of  surfaces  relative  to  the  viewer,  and  the  overall  orientation  of  a  large  surface  such  as  the  ground  seen 
from  the  air.  There  is  another  related  sort  of  information  which  also  seems  primitive  and  important:  the 
direction  of  movement  Whether  running  through  a  forest  or  flying  at  low  altitude,  we  are  visually  aware  of 
which  way  we  arc  travelling  relative  to  the  environment. 

If  we  regard  shape,  orientation,  and  scale  as  three  types  of  3-D  information  necessary  for  flying  relative  to 
the  terrain,  we  may  approach  the  problems  of  improving  the  terrain  simulation  in  these  terms.  The  approach 
is  as  follows. 

11  The  basic  approach 

The  visual  deficiencies  of  the  LLF  simulation  would  be  cast  in  terms  of  shape,  orientation,  and  scale.  In  other 
words,  one  would  ascertain  where  the  performance  problems  tie  in  simulated  LLF  flight,  to  what  extent  the 
cause  is  inadequate  perception  of  the  shape  of  the  terrain  (is  that  ridge  sharp  or  is  it  smoothly  rounded?  what 
does  the  terrain  do  over  to  the  right?),  inadequate  perception  of  orientation  (is  the  aircraft  yawing?  is  the  ground 
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ahead  rising  or  horizontal?),  and  inadequate  perception  of  scale  (how  high  am  I  above  the  ground?  how  far  is 
the  ridge?).  Insight  into  the  underlying  problems  cast  in  these  terms  might  be  gained  by  many  means.  Pilots 
might  even  be  verbally  examined  in  these  terms,  because  we  all  have  strong  (and  similar)  understanding  of 
what  is  meant  by  shape,  orientation,  and  scale.  It  was  mentioned  earlier  that  pilots  commonly  remark  that 
judging  altitude  is  difficult.  That,  for  instance,  is  a  matter  of  scale. 

In  addition  to  direct  interview,  there  should  be  more  quantitative  psychophysical  means  for  determining 
what  3-D  information  is  deficient,  but  again,  cast  only  in  terms  of  shape,  orientation,  and  scale,  without 
attempting  to  uncover  what  "depth  cues"  arc  missing  front  the  CIG  display. 

Note  that  we  seem  to  attack  what  has  been  a  thorny  problem  with  only  very  informal  and  casual 
terminology.  Be  assured  that  the  rigor  comes  later,  when  we  apply  knowledge  about  visual  perception  to 
suggest  improvements  to  scale,  or  whatever.  We  will  see  an  example  of  this  analysis,  when  we  discuss  how 
shape  is  perceived  from  undulating  surface  contours.  ITiis  would  have  importance  in  the  CIG  depiction  of 
rolling  terrain.  First  it  will  be  useful  to  examine  a  wide  range  of  visual  processes  so  that  we  begin  to  sort  out 
which  processes  concern  shape,  which  concern  orientation,  and  so  forth.  In  a  sense,  we  are  abandoning  the 
rather  simple  notion  of  "depth  cue"  for  three  types  of  surface  information,  not  just  depth.  ITiis  has  proven 
useful  in  vision  research  proper.  To  illustrate,  stercopsis  is  not  merely  a  "cue"  to  depth  but  also  of  shape,  at 
least;  shading  also  provides  shape  information  but  tells  us  nothing  about  scale  (the  shading  of  the  moon  is 
similar  to  that  of  an  orange).  We  cannot  expect  a  simple  correspondence  betw  een  process  and  cither  shape,  or 
orientation,  or  scale  --  vision  is  far  too  intertwined  to  be  cast  in  such  simple  terms.  But  on  the  other  hand,  if 
we  know  that  we  need  additional  information  about  shape,  we  have  many  visual  means  for  providing  that 
information. 

Given  understanding  of  where  the  visual  deficiencies  lie,  and  where  we  might  make  improvements,  we 
then  examine  the  associated  perceptual  processes  theoretically.  It  should  be  understood  that  our 
understanding  of  vision  is  not  sufficiently  mature  that  we  might  treat  this  task  as  a  lookup  procedure.  We 
have  no  ready-made  recipes  in  this  regard. 

As  a  final  introductory  remark,  note  that  although  we  might  treat  scale,  orientation  and  shape  as 
substantially  independent  topics,  we  will  probably  find  that  improvements  to  the  CIG  display  intended  to 
enhance,  say,  the  perception  of  shape  will  also  improve  the  apparent  orientation,  and  so  forth.  111050  side 
effects  would  be  welcomed,  of  course.  But  the  general  strategy  is  to  understand  how  one  gains  a  perception  of 
shape,  orientation,  and  scale,  in  order  to  increase  the  effectiveness  of  the  simulator  in  inducing  those 
perceptions. 

2.2  Shape;  what  arc  the  sources? 

There  are  several  potential  sources  of  information  about  the  shape  of  a  surface,  including  binocular  disparity 
from  stercopsis,  visual  motion,  shading,  texture  gradients,  and  surface  contours.  Let  us  consider  each  in  turn. 

First,  although  stercopsis  provides  an  acute  sense  for  3-D  shape  in  our  usual  environment,  it  is  not  useful 
for  large  scale  shape  perception,  e.g.  of  terrain  a  few  miles  away.  The  reason  is  that  only  the  very  near  terrain 
causes  detectable  stereo  disparities,  and  since  that  portion  of  the  scene  is  also  moving  across  the  field  of  view 
at  high  angular  velocity,  one  must  track  some  near  by  surface  feature;  otherwise  it  is  blurred.  On  the  other 
hand,  stercopsis  cannot  be  ruled  out  as  potentially  determining  scale,  especially  in  the  foreground  (sec  later). 

2.2.1  Motion:  parallax,  optic  flow,  and  shear 

Visual  motion  provides  one  of  the  dominant  sources  of  information  about  shape.  Several  different  aspects  of 
motion  have  been  distinguished,  such  as  "shear"  (discontinuities  in  projected  angular  velocity  which  arise,  for 
example,  along  the  edge  of  a  physical  object  seen  against  a  relatively  distant  background),  "optic  flow"  (the 
widc-field  visual  cfTcct  from  movement  through  the  environment),  and  "motion  parallax”  (the  changing 
projection  of  an  object  as  it  moves  relative  to  us,  from  which  we  infer  its  3-D  shape). 

Before  discussing  these  aspects  of  motion,  we  reiterate  that  visual  motion  across  the  CIG  display  comes 
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automatically  when  the  3-D  model  is  transformed  by  perspective  projection  while  "continuously"  (Le.  at  a 
sufficient  rate)  updating  the  viewer’s  position  in  that  model  according  the  speed  and  direction  of  motion 
relative  to  the  model.  Thus  motion  per  se  is  given;  it  is  what  is  in  motion  across  die  CIG  display  that  is 
relevant  to  our  discussion.  For  instance,  arc  moving  dots  sufficient?  (l  uminous  dots  arc  particularly  "cheap" 
to  display  on  many  CIG  systems  while  surface  patches  arc  in  relatively  short  supply.) 

In  fact,  mov  ing  dots  arc  effective;  if  a  collection  of  3-D  points  in  some  rigid  arrangement  (such  as  the 
runway  marker  lights  and  the  lights  of  nearby  buildings  and  streets)  arc  projected  in  motion  on  the  display  as 
if  we  were  moving  relative  to  them  (as  in  landing)  the  visual  interpretation  in  3-D  is  remarkably  precise.1  Of 
course,  the  designers  of  v  isual  simulator  night  displays  have  long  been  familiar  with  this.  Note  the  3-D  shape 
is  accurately  perceived,  but  not  the  scale,  unless  there  arc  familiar  cues  such  as  die  runway  width  (see  later). 

So  we  know  that  points  arc  sufficient  stimuli,  what  about  lines  and  edges?  What  do  diey  contribute  over 
mere  points  of  light?  This  perceptual  issue  has  not  been  fully  unrav  elled,  but  some  relev  ant  observ  ations  may 
be  made.  Our  visual  processes  seem  to  require  distinct  and  traceable  image  points  in  order  to  derive  3-D 
shape,  but  a  curve  or  line  only  offers  the  endpoints  or  points  of  discontinuity  in  tangent;  arbitrary 
intermediate  points  along  the  curve  cannot  be  tracked  (sec  [Wallach  &.  O'Connell  1953]).  Incidentally,  this 
suggests  that  only  the  corner  points  of  the  checkerboard  pattern  (popularly  used  to  depict  terrain)  contributes 
useful  input  to  the  motion  interpretation  process.2 

When  travelling  across  a  textured  surface  diere  is  an  apparent  streaming  of  detail  across  die  visual  field, 
commonly  termed  optic  flow.  This  is  regarded  as  a  wide  field  phenomenon,  while  the  previously  discussed 
motion  parallax  is  detailed  and  fovcal.  What  3-D  information  is  derived  from  optic  flow  the  periphery? 
Some  hypotheses  have  been  forwarded  that  die  optic  flow  specifies  distance  up  to  a  scalar  [Gibson  1950; 
Nakayama  &  Loomis  1974]  and  even  local  surface  orientation  everywhere  across  the  image  [Koendcrink  & 
van  Doom  1976].  Those  two  proposals  suggest  diat  optic  flow  provides  shape  (but  not  scale)  information,  but 
they  arc  largely  theoretical;  it  remains  to  be  seen  what  sort  of  3-D  information  is  derived,  in  fact,  by  the 
human  visual  system.  It  is  possible  that  negligible  shape  information  is  derived  from  die  periphery,  even 
under  the  best  conditions.  Optic  flow,  particularly  in  die  ease  of  l.I.F.  may  only  provide  information  about 
orientation  and  the  direction  of  travel  (this  is  discussed  further  later). 

Another  aspect  of  visual  motion  is  motion  shear.  Whenever  an  opaque  surface  feature  protrudes  above  the 
mean  surface  level,  it  occludes  from  view  that  which  is  behind  it.  Occlusion  dicreby  provides  information 
about  relative  distance  --  it  allow  us  to  partition  the  visual  world  into  surfaces  segregated  by  distance.  This  is 
particularly  striking  in  l.I.F  when  die  relative  motion  of  nearby  ridges  against  the  background  terrain  causes 
them  to  be  seen  in  relief.  Since  shear  contributes  to  the  apparent  terrain  shape,  it  deserves  study.  In 
particular,  the  question  of  what  is  shearing,  whether  it  is  a  simple  dot  patterns  or  edges  or  lines,  is  of 
importance  to  CIG  design.  The  perception  of  3-D  shape  from  motion  was  only  briefly  sketched  here. 
Hopefully  it  shows  dial  while  studying  the  perceptual  processes  one  can  also  derive  specific  implications  for 
CIG  display. 

2.2.2  Shading,  texture  gradients,  and  contours 

We  have  considered  the  dynamic  aspects  of  the  visual  display.  The  other  side  of  die  coin  arc  the  static 
properties  which  carry  information  about  shape.  These  include  shading,  texture  gradients,  and  contours  that 
lie  across  the  surface. 

T  he  apparent  3-D  shape  of  a  surface  is  usually  enhanced  when  shaded.  The  interpretation  of  shading 


I.  Sec  [Ullman  1979]  for  a  theorem  proving  how  this  interpretation  is  feasible  solely  by  analysis  of  the  images  without  requiring  higher 
level  knowledge, 

2  But  that  docs  not  mean  one  can  replace  the  checkerboard  with  dots  (placed  where  each  corner  was)  and  have  an  equally  compelling 

3  0  impression  Ihc  checkerboard  also  provides  a  useful  texture  gradient,  at  least.  This  demonstrates  a  difficult  aspect  of  this  work: 
visual  processes  and  their  visual  inputs  arc  largely  intertwined. 
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information  is  analytically  a  \cry  difficult  problem  requiring  knowledge  of  the  illuminanl  directions,  the 
reflectance  functions  of  the  various  surfaces,  and  means  for  dealing  with  complex  shadowing  and  mutual 
illumination  conditions  [Horn  1975J.  Hut  the  human  visual  system  probably  docs  not  attempt  such  a  solution, 
instead  extracts  weaker  inferences  of  3-1)  shape  that  would  be  true  without  having  to  know  die  lamination 
conditions  and  the  particular  reflectance  properties  of  the  surface.  I  bis  probably  explains  the  success  of  even 
crude  approximations  to  real  shading  that  are  adopted  by  CIG  systems,  further  research  into  human 
perception  of  shading  should  lead  to  understanding  just  how  simple  an  approximation  to  shading  can  still  be 
effective  in  the  simulator  display. 

Another  source  of  3-1)  information  which  we  will  discuss  only  briefly  is  the  texture  gradient.  A 
homogeneous  distribution  of  physical  texture  across  a  surface  results  in  a  texture  gradient  in  die  image, 
because  of  perspective  projection,  the  image  texture  in  every  locality  is  both  foreshortened  and  sealed.  We 
recover  shape  information  (specifically  distance  up  to  a  scalar)  from  die  texture  gradient,  but  as  discussed 
cailier,  probably  not  from  texture  density.  1-urther  research  is  needed  to  determine  how  texture  gradients  are 
measured  in  order  th.it  we  may  gain  insight  into  how  best  to  depict  texture  for  CIG  display. 

Contours  that  lie  across  a  surface,  as  depicted  in  figure  la,  arc  useful  for  inferring  the  shape  of  the  surface. 
To  make  sense  of  die  contours  in  the  image,  however,  we  must  make  certain  assumptions  about  their 
geometry.1  We  will  turn  this  to  our  advantage  by  insuring  that  those  assumptions  arc  not  violated  when 
modelling  terrain  by  means  of  contours. 

Observe  in  figure  la  that  the  lines  appear  drawn  across  an  undulating  surface.  The  basic  assumption  that 


wc  make  is  that  the  curvature  of  the  lines  reflects  surface  curvature.1  It  is  as  if  the  lines  were  straight  lines  on 
a  carpet  that  became  curved  because  the  carpet  was  rippled.  More  formally,  these  contours  arc  geodesics  with 
the  additional  property  of  being  lines  of  greatest  curvature  (see  (Hilbert  &  Cohn-Vosscn  1952]  for  a  lucid 
discussion  of  these  concepts). 

A  line  of  greatest  curvature  may  be  thought  of  as  a  path  across  a  surface  which  experiences  the  greatest 
undulation.  Note  that  in  figure  1  b  the  straight  lines  that  follow  the  ridges  are  perpendicular  to  the  lines  of 
greatest  curvature.  The  straight  lines  arc  lines  of  least  curvature  and  the  surface  is  singly  curved.  The 
perpendicularity  of  the  two  curves  is  very  important  --  it  gives  us  constraint  on  the  shape  of  the  surface. 
Roughly  speaking,  our  interpretation  of  surface  contours  embodies  die  following  geometric  reasoning  (the 
various  deductions  follow  from  theorems  in  Differential  Geometry,  but  they  will  not  he  elaborated  upon 
here). 


(i)  the  physical  curves  arc  lines  of  curvature. 

(ii)  intersecting  curves  meet  at  a  right  angle  on  the  surface  as  a  consequence  of  (i). 

(iii)  where  the  surface  contours  are  parallel  the  surface  is  singly  curved  (like  a 
rippled  carpet). 

(iv)  because  of  (i)  and  (iii)  the  curves  arc  geodesics  and  planar. 

The  above  provides  the  likely  basis  for  our  perception  of  shape  from  surface  contours  in  images  such  as  figure 
1.  It  may  also  act  as  die  basis  for  design  rules  relevant  to  simulators. 

When  modelling  terrain,  especially  undulating  terrain,  it  is  commonplace  to  use  curves  that  resemble  the 
edges  of  fields,  roads,  or  fences.  These  curves  carry  information  about  the  terrain  shape  just  as  figure  1  does, 
but  only  if  they  are  restricted  geometrically  in  the  manner  diat  die  human  visual  system  implicitly  assumes. 
That  is  to  say,  the  curves  that  a  simulator  engineer  lays  in  the  terrain  model  should  have  the  following 
restrictions: 


(i)  each  curve  should  be  planar,  a  line  of  curvature,  and  geodesic. 

(ii)  any  intersecting  curves  should  meet  at  a  right  angle. 

(iii)  if  the  surface  is  singly  *  •■ved,  like  a  smooth  ridge,  it  is  only  necessary  to  depict 
a  few  lines  of  greatest  cur\<j,ure  (as  in  figure  la).  The  lines  parallel  to  the  ridge 
may  be  omitted  in  general. 

(iv)  if  the  surface  is  doubly  curved,  like  a  round  hill,  it  is  important  to  place  lines  of 
curvature  at  close  spacing.  It  is  best  to  include  both  sets  of  lines  of  curvature,  so 
the  surface  appears  to  have  a  net  draped  over  it  (figure  la).  Each  intersection  will 
then  be  a  right  angle,'  and  the  surface  shape  will  be  readily  apparent 

These  are  a  few  rules  which  should  be  followed.  Note  that  they  may  be  "broken"  and  we  might  still  correctly 
perceive  the  surface  from  motion,  just  as  we  can  understand  a  strange  piece  of  sculpture  by  walking  about  it 
and  seeing  it  from  different  perspectives.  For  instance,  suppose  a  ridge  were  depicted  by  a  contour  that 
climbs  over  it,  but  instead  of  following  a  line  of  greatest  curvature  (the  "fall  line”  as  a  skier  would  put  it)  it 
climbs  obliquely  over  the  ridge.  At  a  glance  that  curve  would  mislead  us  --  we  would  see  the  surface  shape 
incorrectly,  but  with  changing  viewpoints  wc  might  later  sec  the  surface  correctly  and  understand  that  the 
curve  is  not  what  we  assumed. 


1.  Note  that  we  do  not  easily  sec  figure  1  for  what  it  is,  a  collection  of  lines  on  a  flat  sheet  of  paper.  Instead,  we  interpret  the  undulation 
as  having  been  caused  by  the  underlying  surface. 
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Depicting  surface  shape  by  undulating  lines,  as  just  discussed,  should  be  compared  with  the  method  of 
using  checkerboards.  First  of  all,  checkerboards  have  been  proposed  because  they  are  relatively  economical 
in  terms  of  edges,  and  squares  seen  in  perspective  are  easily  interpreted  in  3-D.  A  checkerboard  pattern,  in 
fact,  is  probably  optimal  for  depicting  a  planar  surface,  given  the  current  limitations  in  displayablc  edges.  But 
the  straightforward  extension  of  checkerboards  to  undulating  surfaces  may  not  be  the  best  approach.  First  let 
us  consider  singly  curved  surfaces  -  ripples  in  a  carpet.  Checkerboard  patterns  arc  useful  for  ridges  so  long  as 
the  rows  arc  arranged  parallel  with  the  ridge.  In  that  case  we  have  the  geometric  arrangement  of  figure  lb 
where  the  edges  of  the  squares  that  arc  parallel  with  the  ridge  (in  other  words,  those  that  lie  at  constant 
elevation)  are  lines  of  least  curvature,  and  the  other  edges  climb  over  the  ridge.  But  as  stated  earlier,  the  lines 
of  least  curvature  may  probably  be  omitted  because  they  arc  redundant.  In  terms  of  edges,  then,  a  series  of 
stripes,  rather  than  checkers,  would  be  more  economical  and  probably  would  be  equally  effective.  For  a 
doubly  curved  surface  such  as  a  round  hill,  die  problem  becomes  somewhat  more  difficult,  but  solvable  if 
sufficiently  fine  stripes  or  checkers  arc  used.  That  is  because  a  doubly  curved  surface,  if  sliced  fine  enough, 
can  be  treated  as  singly  curved  in  general.  Incidentally,  the  line  defined  by  a  column  or  row  of  checker 
squares  has  alternating  contrast  along  its  length,  litis  tends  to  break  up  the  lines,  for  the  visual  system  has 
difficulty  in  aggregating  into  a  line  elements  with  opposite  contrast  sense.  ITtc  checkerboard  pattern  is 
unlikely  treated  as  a  collection  of  long  surface  contours,  therefore.  Instead,  each  white  or  black  square  is  seen 
disjointly.  For  that  reason,  long  stripes  or  lines  may  be  more  useful  in  depicting  undulating  terrain. 

In  sum.  parallel  undulating  lines  in  an  image  provide  a  dramatic  sense  of  3-D  shape,  but  to  be  useful  they 
must  correspond  to  3-D  lines  that  have  particular  geometric  restrictions.  So,  provided  these  restrictions  are 
met,  we  have  a  powerful  method  for  depicting  curved  surfaces  -  a  method  which  may  be  more  economical 
and  more  effective  than  die  traditional  checkerboard  patterns. 

2.3  Sorces  of  information  about  orientation 

The  previous  discussion  covered  the  major  sources  of  visual  information  about  surface  shape.  Next  is  for  us 
to  discuss  orientation.  TTiis  discussion  is  more  difficult,  because  3-D  shape  is  most  naturally  described  relative 
to  the  viewer  and  therefore  in  representing  shape  one  simultaneously  captures  several  aspects  of  orientation. 
For  instance,  recall  that  in  the  night  landing  simulation,  motion  parallax  involving  mere  luminous  dots  on  a 
dark  background  can  give  a  compelling  impression  of  movement  towards  die  solid  earth  during  landing. 
Clearly  we  perceive  the  orientation  of  the  terrain  as  well  as  its  shape.  While  orientadon  and  shape  are 
intertwined,  it  is  useful  to  approach  visual  processing  from  one  or  the  other  perspective,  depending  on  the 
need. 

The  direction  of  movement  and  one's  spatial  orientation  are  two  forms  of  orientation  information  that  are 
relatively  distinct  from  shape.  Both  are  difficult  to  visually  determine  in  LLF  since  uneven  terrain  often 
prevents  one  from  using  a  distant  horizon  as  a  attitude  reference.  But  optic  flow  seems  useful  [Gibson  1950J. 
particularly  in  IJ.F,  where  the  terrain  appears  smeared  and  blurred  (see  [Marker  &  Jones  1980]).  However, 
the  highly  simplified  CIG  displays  might  not  provide  sufficiently  dense  surface  texture  moving  with  sufficient 
smoothness  of  motion  to  be  effective.  Again,  we  see  that  further  research  is  needed  to  see  how  optic  flow  is 
visually  processed.  This  research  problem  is  of  particular  importance  to  Ll.F.  A  final  point  regarding  optic 
flow  should  be  made.  Simply  because  the  ground  texture  appears  blurred  and  streaky  in  LLF  docs  not  mean 
that  one  can  display  a  simple  pattern  of  streaks  in  the  periphery  displays.  When  one  tracks  a  detail  on  the 
ground,  it  becomes  roughly  stationary  on  the  retina  and  the  smearing  of  detail  vanishes.  We  do  not  know 
what  role  tracking  eye  movements  would  play  in  LLF  (certainly  they  arc  important  when  scrutinizing  ground 
targets,  but  what  importance  they  have  in  the  basic  problem  of  flying  LLF  is  not  yet  known). 

A  local  "cue"  to  orientation  which  has  been  incorporated  in  CIG  displays  is  the  simple  geometric 
arrangement  of  two  lines  or  edges  that  intersect  at  right  angles  on  the  surface  (figure  2a).  It  is  straightforward 
to  show  that  the  image  of  a  right  angle  (which  because  of  foreshortening  in  the  perspective  projection  appears 
as  an  obtuse  angle)  carries  some  information  about  the  surface  orientation,  but  it  alone  docs  not  specify  a 
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Figure  1  The  intersecting  lines  in  a  appear  to  lie  on  planar  surfaces  in  3-D  The  interpretation  involves  assuming  the  intersection  is 
perpendicular  and  the  two  lines  arc  of  equal  length  on  the  surface.  The  intersections  in  b  may  be  thought  of  as  a  signpost  at  a  street 
intersection  seen  from  different  viewpoints  To  interprets  this  configuration  it  is  only  necessary  to  assume  the  intersecting  lines  are 
mutually  perpendicular  in  3-D. 


unique  3-D  orientation;  it  could  theoretically  correspond  to  a  large  variety  of  differently  oriented  planes  in 
3-D.  Additional  constraint  comes  from  assuming  that  the  two  lines  arc  equal  length  in  3-D.1  Note  that  these 
two  constaints  -  perpendicular  intersections  and  equal  length  segments  --  are  neatly  combined  in  the  square 
checkerboard  patterns  which  arc  often  used  in  terrain  simulation.  Perpendicular  intersections  in  general  are 
useful  indicators  of  surface  orientation,  provided  the  lines  or  edges  arc  of  equal  length  on  the  surface. 

A  point  that  has  seldom  been  emphasized  is  that  a  trihedral  intersection,  three  mutually  perpendicular 
lines  (figure  2b),  actually  carries  enough  information  in  its  image  to  fix  its  surface  orientation  precisely.2  The 
arrangement  resembles  a  signpost  at  a  street  intersection  -  note  that  the  line  segments  no  longer  need  to  be 
equal  length  in  3-D.  this  configuration  would  be  a  very  effective  means  for  indicating  orientation,  even 
scattered  across  otherwise  featureless  terrain. 

2.4  Sources  of  information  about  scale 

Now  let  us  suppose  that  the  terrain  simulation  is  adequate  in  terms  of  shape  and  orientation.  The  problem 
that  would  remain  is  to  insure  that  he  can  judge  the  distance  to  the  surface  and  his  altitude  above  it  with 
sufficient  accuracy.  What  arc  the  sources  of  information  about  scale  that  he  might  utilize?  Scale  may  be 
inferred  by  several  perceptual  methods:  directly  from  stcrcopsis,  or  indirectly  from  the  known  size  of 


1.  There  is  slill  an  ambiguity  of  the  Ncckcr  cube  variety.  But  when  viewing  this  configuration  within  the  larger  context  of  the  ground 
plane,  that  ambiguity  is  resolved. 

2.  Again,  up  to  a  Ncckcr  reversal  which  can  be  disregarded  here. 
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recognizable  objects,  from  known  velocity,  and  several  other  methods  that  we  will  discuss.  Scale  is  certainly 
necessary  for  various  flying  tasks;  the  difficulty  dial  pilots  often  have  in  landing  amphibious  planes  on  water 
and  maintaining  low  level  flight  over  water  and  sand  dunes  suggests  that  there  are  real  visual  scenes  that  are 
deficient  in  information  about  scale. 

2.4.1  Slereopsis 

Stcrcopsis  may.  in  theory,  have  a  minor  contribution  at  low  altitude,  but  to  see  this  we  need  to  be  somewhat 
quantitative  for  a  moment.  A  generous  estimate  of  the  maximum  distance  which  may  be  distinguished  from 
infinity  by  stcrcopsis  is  on  the  order  of  1800  ft.  (computed  from  a  slcrcoacuity  of  24  arc  seconds  {Graham,  et 
at.  1949J).  Objects  would  be  seen  "in  relief  against  the  distant  horizon  only  when  significantly  closer,  say 
1000  ft.  Of  course,  stcrcopsis  is  most  useful  in  the  very  near  environment,  say  within  30  ft.,  but  since  we  arc 
sensitive  to  stereo  disparity  at  larger  distances,  it  might  serve  a  useful  purpose  in  determining  die  size  and 
distance  of  somewhat  more  distant  objects.  If  that  is  the  ease,  stcrcopsis  might  give  the  pilot  a  sense  of  his 
altitude  when  flying  I.I.F  (some  pilots  reportedly  look  out  of  die  side  canopy  to  judge  dicir  altitude  [Marker  & 
Jones  1980)).  We  examine  diat  hypodiesis  here. 

At  roughly  100  foot  altitude  the  pilot  can  see  downward  sufficiently  well  to  be  viewing  surface  detail  within 
the  useful  range  of  stcrcopsis  [Kennedy  &  McKcchnic  1970),  but  because  of  the  blurring  and  streaking  which 
we  have  discussed,  unless  the  pilot  tracks  some  feature  on  the  ground,  stcrcopsis  would  probably  fail.  But  a 
significant  impression  of  depth  might  be  derived,  provided  one  tracks  a  surface  feature  for  sufficient  time  to 
achieve  stcrcopsis  before  it  is  out  of  view.  Consider  an  aircraft  travelling  500  kt.  at  100  ft.,  and  a  surface 
feature  that  passes  within  500  ft.  of  the  aircraft  at  nearest  approach.  One  may  easily  compute  that 
approximately  two  seconds  elapses  from  the  time  the  point  enters  stereo  range  (1800  ft.)  to  when  it  passes 
directly  by  die  side  of  the  aircraft,  l  ienee  if  it  is  still  tracked  after  it  is  passed  by,  one  has  a  reasonably  long 
time  to  judge  its  distance  by  stcrcopsis.  Thus  it  is  possible,  in  theory,  diat  stcrcopsis  directly  provides  a  sense 
of  scale. 

There  is  evidence  that  casts  doubt  on  this  hypothesis,  however.  The  evidence  is  simple:  recall  that  certain 
types  of  terrain  arc  particularly  difficult  to  fly  over  at  low  level  -•  specifically,  sand  dunes  and  open  water. 
There  is  sufficient  viewing  time  to  achieve  binocular  fusion,  sufficient  visible  detail  on  which  to  establish 
stereo  fusion,  and  a  large  region  of  the  surface  is  within  stereo  range  at  low  altitude.  Nonetheless  LLF  is 
difficult  over  that  terrain.  Apparently  die  sense  of  scale  (if  any)  that  one  derives  from  stcrcopsis  is  insufficient 
for  making  the  critical  altitude  judgments.  Stated  another  way,  stcrcopsis  may  play  some  role  in  I.I.F,  but  it  is 
probably  insignificant.  It  must  be  said,  however,  that  the  above  is  not  conclusive  --  the  issue  deserves  carefiil 
experimental  consideration. 

2.4.2  Known  size 

One  method  to  infer  distance  is  from  the  measured  retinal  size  of  an  object  of  known  size.  It  is  well  known 
that  the  distance  to  an  object  varies  inversely  with  the  angle  it  subtends.  Theoretically,  if  one  knew  that  a  tree 
were  20  ft.  high,  one  could  also  know  its  absolute  distance  one  the  basis  of  its  retinal  size.  Certainly  the  visual 
system  capitalizes  on  this  relation,  and  we  even  consciously  search  a  novel  scene  for  some  cue  to  its  scale  ( n.b. 
conscious  attention  is  not  required,  as  Helmholtz  [1925]  recognized  in  his  terming  it  "unconscious  inference"). 
There  arc  phenomena  which  strongly  suggest  that  we  do  infer  distance  from  known  size  (c.g..[Enright  1970]). 
however,  empirical  studies  of  "size  constancy"  and  absolute  distance  perception  [Rock  &  McDermott  1964; 
Epstein  &  Landauer  1969;  Gogel  1971;  Hochbcrg  1971]  have  shown  that  the  psychophysical  relation  between 
distance  and  retinal  size  is  not  as  simple  as  might  be  expected.  'This  is  one  place  where  simulator  designers 
have  experienced  difficulty  in  interpreting  the  psychological  literature.  Many  of  the  experiments  were 
performed  in  artificially  rcstictcd  viewing  situations  (eg.,  darkened  rooms  with  few  reference  objects)  and 
those  that  were  performed  "in  the  field"  would  involve  verbal  judgments  of  distance,  eg.,  as  a  target  being 
so-many  feet  away  [Gibson  &  Bergman  1954]  or  comparison  judgments  between  two  distances  [Foley  1972). 
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Few  experiments  reveal  just  how  precisely  we  perceive  the  scale  of  the  visual  world  from  objects  of  known 
si/.c. 

It  must  be  stressed  that  we  are  ultimately  concerned  w  ith  die  visual  judgments  of  absolute  distances,  and 
therefore,  with  providing  sufficient  information  so  that  the  pilot  can  fly  100  ft.  above  the  terrain,  for  instance. 
The  known-si/e  method  probably  plays  a  role  in  this  ability.  Ibis  is  another  place  where  tightly-directed 
investigation  is  critically  needed. 

2.4.3  Known  velocity 

A  potentially  useful  method  for  computing  scale  is  quite  similar  to  the  known-si/c  method  just  discussed.  If 
one  knew  the  velocity  of  travel  past  an  object,  one  could  infer  the  distance  to  that  object  from  the  induced 
angular  velocity  (<\g..  retinal  velocity  given  a  fixed  ga/e.  or  velocity  relative  to  sonic  point  on  the  cockpit 
canopy).  The  following  relation  between  absolute  distance  </.  absolute  velocity  I',  and  angular  velocity  u>  was 
pointed  out  by  Gibson  and  others  (e.g.,  [Gibson  1950:  Nakayama& Loomis  1974]): 

.  _  l'sin/3 
—  u 

where  /?  is  the  angle  from  the  direction  of  travel  to  the  given  point  whose  distance  is  to  be  measured.  The 
relation  presumes  that  the  given  point  is  stationary  and  the  viewer  is  in  pure  translation.  It  is  powerful  in  that 
it  allows  one  to  compute  a  depth  map  for  die  entire  visual  field  (everywhere  dierc  is  detail  moving  across  the 
field  of  v  iew)  in  terms  of  oi  and  fl  (measurable,  in  theory )  and  the  single  unknown  V.  That  dcpdi  map  would 
specify  absolute  distance  if  die  absolute  velocity  I  were  known. 

Conscious  deduction  is  not  necessary  or  even  likely  in  this  process.  Instead,  a  pilot  experienced  in  flight  at 
low  altitude  and  high  velocity  might  come  to  expect  a  particular  angular  velocity  in  die  periphery,  in  a  manner 
analogous  to  driving  a  car  and  expecting  die  road  to  slip  by  at  an  appropriate  rate  for  any  given  driving 
speed.1  Then  if  the  groundspeed  is  held  constant  but  die  altitude  is  lower  than  normal,  the  angular  velocity 
would  be  higher  than  normal,  and  vice  verso.  This  relation  could,  dicorclically,  uccounGbr  die  precision  with 
which  altitude  may  be  held  in  1.1  .F  by  experienced  pilots.  It  is  not  inconceivable  dial  part  of  die  skill  that  a 
pilot  acquires  through  flight  training  is  the  unconscious  calibration  of  retinal  velocity  according  to  ground 
speed  and  altitude,  and  furthermore  the  development  of  an  effective  feedback  loop  that  attempts  to  control 
altitude  according  to  actual  retinal  velocity  at  any  instant. 

It  should  be  relatively  straightforward  to  establish  whether  this  is  the  ease.  The  direct  relationship  between 
altitude  and  groundspeed  predicts  dial  as  speed  increases  dierc  should  be  a  tendency  to  increase  the  cruise 
altitude,  other  factors  being  constant,  flic  following  evidence  [J.  Richter,  personal  communication]  suggests 
that  this  occurs.  Interestingly,  it  is  most  apparent  when  flying  over  open  water  (the  situation  w'C  noted  before 
as  difficult).  Perhaps  die  fact  dial  water  provides  little  evidence  of  scale,  compared  to  a  richly  textured  rural 
terrain  containing  familiar  surface  features,  allows  one  to  observe  the  weaker  contribution  of  the  known  speed 
method.  Over  the  ground,  one  s  altitude  tends  to  remain  fairly  constant  as  speed  increases,  but  over  water  the 
altitude  definitely  tends  to  climb  as  the  aircraft  is  accelerated.  It  is  consistent  with  there  being  some 
unconscious  process  attempting  to  maintain  an  expected  angular  velocity,  where  an  increase  in  angular 
velocity  it  is  attributed  to  a  loss  of  altitude  rather  than  an  increase  in  ground  speed.  These  informal  reports 
suggest  that  the  known  velocity  mcdiod  for  computing  scale  may  have  some  validity  in  actual  flight,  and 
therefore  should  be  given  careful  experimental  investigation.  The  experiments  should  probably  be  performed 
using  unfamiliar  surface  textures  to  examine  the  relationship  between  speed  and  altitude.  The  problem  must 
be  addressed  in  conjunction  with  an  investigation  of  texture,  as  there  is  undoubtedly  the  effect  will  vary  in 
magnitude  with  texture  variations. 


1  It  is  noteworthy  that  when  changing  front  a  passenger  car  to  a  low  sports  car  the  apparent  speed  is  greater  merely  because  of  the  larger 
angular  velocities  associated  with  being  nearer  the  ground. 
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2.4.4  Other  methods 


In  addition  to  the  two  methods  just  discussed,  there  arc  other  geometrical  methods  for  gaining  scale 
information.  Several  have  been  discussed  by  Marker  and  Jones  [1980],  Typically  they  consist  of  some 
geometric  quantity  --  e.g.,  the  ratio  of  angle  subtended  by  some  vertical  surface  feature  compared  to  its 
angular  distance  below  the  horizon  --  from  which  one  may  judge  whether  the  altitude  is  maintained  constant, 
or  whether  the  aircraft  will  clear  a  vertical  obstacle.1  While  strictly  speaking  these  geometric  quantities  do  not 
specify  scale  in  the  sense  we  mean,  they  arc,  nonetheless,  potentially  useful  in  flying.  We  probably  should 
distinguish  between  qualitative  problems  that  can  be  solved  geometrically  (such  as  obstacle  clearance  and 
maintaining  constant  altitude)  and  quantitative  problems  (such  as  clearing  obstacles  by  so  many  feet, 
maintaining  a  [kirticular  altitude,  and  so  forth).  But  it  is  not  certain  which  sort  of  problem,  die  qualitative  or 
the  quantitative,  deserves  the  greater  attention  at  the  moment. 

3  Concluding  remarks 

Three  types  of  3-1)  infonnation  that  must  be  gathered  by  the  visual  system  in  order  that  a  pilot  might  fly  close 
to  the  terrain  arc  its  scale,  shape,  and  orientation.  The  visual  deficiencies  of  the  1.1.1'  simulation  can  be 
discussed  in  such  terms.  Indications  arc  that  scale  information  requires  the  greatest  improvement. 

More  rigrous  analysis  comes  next.  We  gain  insight  into  improving  the  display  by  learning  how  the  human 
visual  system  determines  scale  (for  example).  1'herc  arc  many  sources  of  scale  information,  but  much  more 
effort  is  required  before  we  can  make  concrete  suggestions  regarding  CIG  improvements. 

We  did  sec  an  example  of  where  the  computational  analysis  of  surface  contours,  a  source  of  shape 
information,  leads  to  rather  specific  suggestions  regarding  die  visual  display.  For  our  visual  systems  to  make 
sense  of  image  curves  in  terms  of  actual  contours  lying  across  physical  surfaces,  a  number  of  geometric 
assumptions  have  to  be  made.  Analyzing  what  these  might  be,  in  theory,  coupled  with  psychophysical 
verification  that  those  particular  assumptions  arc  involved  in  our  interpretation  process,  and  not  some  other 
set  of  assumptions,  leads  to  some  design  rules,  as  it  were.  The  design  rules  arc  intended  for  the  simulator 
engineer  who  depicts  terrain  using  curves  (one  curve  might  be  meant  to  depict  a  fence  over  a  hill,  another 
might  depict  the  edge  of  a  field).  It  is  of  course  important  diat  the  pilot  perceive  the  simulated  world  in  the 
way  that  it  was  intended  by  die  designer.  To  do  so,  die  contours  placed  in  the  terrain  model  should  be 
restricted  in  their  geometry  so  dial  they  match  the  geometric  assumptions  that  govern  their  visual 
interpretation. 

ITic  surface  contour  example  is  a  particularly  clean  demonstration  of  how  understanding  some  aspect  of 
vision  has  application  to  simulation.  As  should  be  evident  from  die  previous  discussions,  much  needs  to  be 
learned  about  vision.  But  the  sorts  of  questions  that  we  ask  can  be  efficiently  guided  by  applying  the 
computational  mcdiodology. 

Acknowledgments 

A  number  of  people  made  important  contributions  and  suggestions:  Prof.  W.  Richards  and  Drs.  S.  Collyer, 
I).  Regan,  J.  Richter,  and  S.  Ullman.  The  author  gratefully  acknowledges  the  useful  comments  on  an  earlier 
draft  of  this  article  provided  by  Drs.  K.  Dismukes,  R.  Haber,  and  J.  Hochbcrg.  The  manuscript  was  prepared 
at  the  Artificial  Intelligence  Laboratory  of  the  Massachusetts  Institute  of  Technology,  with  support  provided 
in  part  by  the  Advanced  Research  Projects  Agency  of  the  Department  of  Defense  under  Office  of  Naval 
Research  contract  N00014-75-C-0643  and  in  part  by  the  AI  'OSR  and  NSK  under  MCS79-23110. 
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ABSTRACT 


In  the  first  successful  application  of  Vought's  CAPTV  technology  the 
Navy  has  been  provided  a  unique  training  capability  for  the  forward  looking 
infrared  receiver  (FLIR)  system  utilizing  real  world  imagery.  The  CAPTV 
system  employs  monochrome  aerial  photographs  stored  in  a  large  random  access 
video  data  base,  which,  through  computer  processing,  provides  smooth  high- 
detail  simulated  visual  motion  cues  to  the  pilot  trainee. 

I.  INTRODUCTION 

Flight  simulators  have  come  to  be  recognized  as  important  training 
components  for  several  reasons.  The  rising  cost  of  fuel  and  equipment, 
congestion  over  airports,  the  need  to  avoid  accidents  during  training  are  some 
of  the  more  important  reasons.  Computer  Generated  Imagery  (CGI)  is  presently 
the  most  popular  method  of  image  generation  for  the  visual  systems  in  most 
flight  simulation1.  This  paper  describes  an  alternative  to  CGI  in  that 
aerial  photography  of  the  gaming  area  is  used  instead  of  CGI  as  the  database 
in  our  system.  The  approach  achieves  excellent  scene  realism  for  the  student 
pilot  besides  providing  the  visual  cues  needed  for  most  training  requirements. 
The  photographic  database  is  a  part  of  the  unique  Vought-deve loped  concept 
called  computer  animated  photographic  terrain  view  (CAPTV). 

The  development  of  CAPTV  has  been  a  continuing  effort  by  Vaught  in  the 
last  decade.  Early  systems  were  restricted  by  the  unavailability  of  video 
bulk  storage  with  fast  random  access.  With  the  availability  of  the  video 
disc,  Vought  has  pioneered  applications  for  simulated  visual  displays  using 
oblique  aerial  photography.  The  recent  delivery  of  two  systems  to  the  U.S. 
Navy  to  provide  simulated  Forward  Looking  Infrared  (FLIR)  imagery  represents 
the  first  successful  application  of  the  CAPTV  concept  and  the  use  of  video 
disc. 


The  FLIR  simulators  have  been  added  to  the  Navy's  A-7E  Weapon  System 
Trainers  (WST)  (Device  2F111)  at  Cecil  Field  NAS  in  Florida  and  Lemoore  NAS 
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in  California.  The  real-time  playback  system, when  added  to  the  existing  WST 
equipment, provides  the  student  pilot  with  a  continuous  FLIR  display  of  the 
gaining  area  as  presented  on  the  head-up  display  (HUD)  in  the  cockpit.  The 
scope  of  the  gaming  area  is  large  enough  and  flexible  enough  for  a  simulator 
pilot  to  fly  a  mission  with  total  maneuvering  freedom  within  the  gaming  area. 

The  derived  FLIR  scene  can  be  selected  in  a  12-degree  or  a  magnified  3-degree 
field  of  view.  Finally,  FLIR  registration  with  the  radar  display  in  the  WST 
is  maintained  regardless  of  the  simulated  aircraft  maneuvers  throughout  the 
flight. 

II.  CAPTV  CONCEPT 

The  basic  CAPTV  concept  assumes  an  array  of  still  photographs  taken 
from  an  airplane  that  covers  the  gaming  area  defined  by  pilot  training 
requirements.  Photographs  are  taken  at  regular  intervals  along  straight 
and/or  cross  tracks,  the  interval  between  the  photos  is  proportional  to  the 
altitude.  These  photos  are  scanned,  formatted,  and  stored  in  a  bulk  storage 
device.  As  these  photos  are  retrieved  from  the  storage  medium  for  display, 
the  pilot  'flies'  through  them  using  controls  similar  to  the  ones  on  the 
cockpit  panels.  At  any  given  instant,  knowing  the  pilot's  eyepoint  in 
space,  the  photo  in  the  database  nearest  his  location  is  stretched,  skewed, 
rotated  and  translated  in  a  mathematical  transformation  such  that  the  trans¬ 
formed  photo  would  overlay  a  different  photo  taken  from  the  pilot's  eyepoint. 

This  process  is  a  continuous  one  each  frame  time  under  computer 
control  and  allows  the  introduction  of  smooth  translation  into  a  basically 
still  picture  set  for  any  direction  of  travel  through  that  set.  During  the 
period  in  which  translation  is  taking  place  using  a  given  fixed  scene,  the  com¬ 
puter  is  fetching  a  new  appropriate  view  to  be  used  as  an  overlay  substitute. 

The  new  view  is  selected  by  a  sophisticated  prediction  scheme  that  determines 
when  the  present  photo  must  be  discarded  in  favor  of  a  new  scene.  It  is 
important  to  understand  that  the  transformation  process  does  not  cause  unrealistic 
distortions  of  the  viewed  scene.  The  key  to  CAPTV  lies  in  its  unique  capability 
to  make  one  fixed  photo  serve  in  a  dynamic  translation  situation  long  enough 
to  fetch  another  view  and  incidentally  perform  many  other  functions  as  well. 

III.  OFF-LINE  IMAGE  GENERATION 

Hie  off-line  image  generation  system  of  the  FLIR  simulation  (Fig.  l) 
was  used  to  develop  the  database  for  the  playback  system.  The  image  generation 
consisted  of  the  following  subsystems: 

(a)  Aerial  Camera:  A  Z less /Has selb lad  38mm  Biogon  lens  using  70mm  Infrared 
film  rigidly  attached  to  an  airplane  that  flew  over  two  gaming  areas  (one  on  the 
West  coast  and  one  on  the  East  coast).  Each  of  the  two  gaming  areas  was  parti¬ 
tioned  to  contain  some  3000  to  4000  'eyepoints'  distributed  into  several 
altitudes  in  straight  and  cross  tracks. 
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Each  of  the  photographs  has  a  FOV  of  60  degrees  in  both  azimuth  and  elevation. 
Objects  w^ich  moved  as  the  terrain  was  photographed  (e.g.,  vehicles  on  the 
road)  were  treated  as  stationary  anomalies. 

(b)  Record  System;  The  flying  spot  scanner  was  used  to  scan  the  57nm 
square  usable  area  of  the  70mm  frames  to  provide  a  pixel  resolution  of 
approximately  4000  x  4000  elements.  The  resulting  analog  video  was  uniquely 
formatted  using  the  Video  Processor  System  (VPS)  such  that  each  terrain  scene 
was  made  up  of  (typically  72)  NTSC  frames.  Appropriate  sync  signals  were 
inserted  to  make  the  signals  suitable  for  recording  on  video  tape  recorders 
and  disks .  During  the  recording  process ,  opportunity  was  afforded  for 
manipulating  the  video  gain  and  bias  of  each  scene  to  correct  anomalies  resulting 
from  earlier  photographic  processes. 

(c)  Film  Edit  System;  This  system  consisted  of  a  digitizer  and  an 
analytical  software  package  residing  on  the  Harris  Slash  6  digital  computer 
(32K,  24  bit)  that  determined  the  approximate  exposure  station  orientations 
(eyepoints)  of  each  photograph.  The  calculations  aided  in  the  proper 
sequencing  of  the  scanned  scenes  on  the  tape  recorder  and  eventually  facilitated 
fast  random  access  of  the  scenes  from  the  video  disks  during  playback. 

(d)  Video  Disk;  The  database  recorded  on  the  Ampex  VPR-1  recorder  was 
transferred  by  MCA  to  video  disks  suitable  for  playing  on  the  MCA  DiscoVision 
Model  700  industrial  optical  video  disk  player^.  Each  of  the  video  disks  can 
hold  up  to  54000  standard  NTSC  frames  -  typically  600  FLIR  scenes.  Both  frame 
identification  numbers  and  Society  of  Motion  Pictures  and  Television  Engineers 
(SMPTE)  codes  were  used  for  the  proper  sequencing  of  the  frames.  Such 
sequencing  is  optimized  to  provide  minimal  search  times  during  playback. 

IV.  ON-LINE  PLAYBACK  SYSTEM 


The  deliverable  real-time  on-line  playback  system  (Fig.  2)  consisted  of 
the  following  sub-systems : 

(a)  Video  Storage  System:  Analog  storage  using  television  techniques 
is  a  practical  and  economical  alternative  to  digital  storage  and  is  especially 
suited  to  pictorial  data.  The  purpose  of  the  storage  system  consisting  of  the 
four  video  disks  was  to  provide  large  segments  of  the  desired  data  under  control 
of  the  host  computer  and  in  anticipation  of  the  real  time  needs  of  the  simulation. 
The  bulk  data  was  distributed  over  the  four  disks  to  optimize  the  retrieval  of 
the  desired  data.  Any  randomly  accessed  TV  frame  on  a  single  play-back  unit 
could  be  located  and  made  available  in  2  to  8  secs.  Adjacent  or  sequential 
frames  required  much  less  time  than  this,  however.  The  players  operated 
essentially  independently  and  were  rotationally  synchronized  to  the  system  timing 
references  for  color  subcarrier  and  horizontal/vertical  raster  scans  (although 
the  entire  system  was  essentially  monochrome,  a  color  subcarrier  was  provided 
for  synchronization  and  clocking  within  the  VPS).  In  operation,  these  units 
were  not  supplying  data  at  the  same  time.  Data  was  transferred  from  only  one 
player  at  a  time  while  the  others  were  searched  for  the  anticipated  subsequent 
scenes. 


29 


r 


(b)  Video  Processor  System:  A s  the  pilot  determined  the  location  and 
FOV  of  the  database  that  he  wanted  to  'fly',  the  host  computer  CPU  transferred 
an  entire  scene  or  a  programmed  fraction  of  the  total  scene  to  the  video 
processor  system.  The  analog  video  was  stripped  off  of  the  NTSC  sync,  and 
was  reformatted  into  the  original  FSS  format.  This  signal  was  then  converted 
into  a  digital  stream  of  8-bit  pixels  and  stored  in  a  3-line  internal  storage. 
Based  on  the  pilot's  zoom  rate  and  the  resulting  requirements  on  the  HUD  FOV, 
the  video  lines  were  appropriately  interpolated  on  in  the  VPS.  The  resulting 
video  was  converted  back  into  the  analog  form  and  under  CPU  control  transferred 
to  the  scan  conversion  system. 

(c)  Scan  Converter  System:  Hiis  system  consisted  essentially  of  two 
analog  storage  tubes.  To  achieve  the  requirement  of  the  continuous  display  of 
video  information,  the  two  scan  converters  were  utilized  in  a  'ping-pong' 
configuration;  that  is,  while  video  information  was  being  read  from  one  scan 
converter,  new  video  was  being  written  on  the  other  scan  converter.  Switching 
then  enabled  the  newly  written  information  to  be  read  from  the  second  scan 
converter  while  the  older  information  on  the  first  scan  converter  was  erased 
and  replaced. 

(d)  Write  Raster:  The  usable  scan  converter  memory  had  a  normalized 
resolution  of  only  1200  lines  of  scanned  FLIP  video.  The  available  resolution 
on  the  disk  for  each  scene  was  approximately  4000  such  lines.  Therefore,  a 
different  write  raster  was  obtained  using  a  2-dimensional  re-sampling  technique 
based  on  the  slew  rate,  FOV  to  be  displayed  and  the  velocity  of  the  aircraft. 

(e)  Read  Raster:  The  Read  Raster  size  was  a  fraction  of  the  Write  Raster 
size  (Fig.  3)  and  this  fact  enabled  the  pilot  to  maneuver  over  a  gaming  area 
for  a  few  seconds  before  he  out-stepped  the  edge  of  the  scan  converter  image. 

A  new  read  raster  was  calculated  every  l/30th  of  a  second  (Fig.  4)  by  generating 
a  (3x3)  transformation  matrix  which  was  used  to  skew  and  rotate  a  window 
off  of  the  scan  converter  to  obtain  the  off-nominal  image. 

(f)  Display :  An  875  line  Conrac  CRT  display  was  used  to  display  the 
FUR  images.  A  joystick  and  a  switch  box  enabled  the  pilot  to  move  into 
different  modes  of  operation  and  flight.  The  FLIR  simulator  display  was 
compatible  with  the  FLIR  imagery  coming  off  the  pod.  The  entire  setup  (Fig.  5) 
was  designed  to  be  added  to  the  existing  A-7E  WST  hardware  at  Cecil  Field,  Florida 
and  Lemoore,  California. 

V.  EXPANDABILITY  CF  DATABASE 

One  of  the  major  advantages  of  the  FLIR  simulator  system  described  above 
is  the  almost  unlimited  expandability  of  the  database  without  changing  the 
record/playback  hardware.  The  terrain  database  delivered  to  the  Navy  was  in 
fact  expanded  to  include  a  database  consisting  of  'enemy'  ships.  This  was 
accomplished  by  taking  photographs  of  a  model  ship  with  enough  density  of 
photographs  to  perform  an  efficient  eyepoint  transformation.  As  explained 
earlier,  the  addition  of  databases  involves  taking  aerial  or  model  photos, 
scanning  on  the  FSS,  formatting,  recording  on  the  video  tape  recorder,  and 
video  disc  mastering.  A  FLIR  record  system  rack  is  permanently  stationed 
in-house  to  accomplish  this  task.  Thus,  there  are  virtually  no  limits  to  the 
expansion  of  database. 
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PHOTOGRAMMETRY 


Riotogrammetry  is  the  science  of  obtaining  extremely  precise  information 
about  physical  objects  and  the  environment  through  the  process  of  recording, 
measuring  and  interpreting  photographic  images3.  Extensive  use  of  photogram- 
metry  was  made  in  the  FLIR  simulator  to  pin-point  accurately  the  eyepoints  of 
the  database  photographs.  The  overlap  between  successive  database  photos  was 
utilized  to  carefully  pick  pass  points  off  the  photos  of  discernible  physical 
features  of  the  terrain,  and  control  points  on  the  terrain  obtained  from  the 
U.  S.  Geological  Survey.  Typical  physical  features  were  street  intersections, 
building  corners,  edges  of  runways,  etc.  Sophisticated  photogrammetric 
equipment  also  allows  common  points  to  be  picked  even  where  there  are  not 
many  discernible  physical  features  on  the  terrain.  The  pass  points  and  control 
points  were  used  as  input  data  to  a  data  reduction  software  package  which 
calculated  the  eyepoint  parameters  to  the  desired  accuracy. 

VII.  SUMMARY  AM)  CONCLUSIONS 


The  Vought-developed  CAPTV  concept  has  resulted  in  a  FLIR  visual  system 
for  the  A-7E  VJST  simulator  that  has  proved  to  be  a  valuable  training  device. 
CAPTV  offers  some  major  advantages  in  its  ability  to  provide  scene-realism, 
detail,  3-diraensionality,  and  texture  that  is  limited  only  by  photography 
and  almost  unlimited  for  expandability  of  database.  The  system  components 
Include  such  state-of-the-art  technology  as  video  discs  and  some  of  the  best 
photogrammetric  software  packages  for  the  preparation  of  the  database.  The 
FLIR  simulator  provides  an  875-line  display  that  is  presently  used  by  the 
Navy  in  full  mission  training  application.  As  a  follow-on  effort,  work  is  In 
progress  at  Vought  to  provide  a  full  color  out-th e-window  visual  system  using 
the  principles  proved  by  the  FLIR  simulation. 
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Abstract 

The  purpose  of  this  study  was  to  test  the  feasibility  of  using  the 
Advanced  Simulator  for  Pilot  Training  (ASPT)  in  training  pilots  for 
combat  in  a  simulated  hostile  environment.  A  combat  environment  was 
developed  in  which  the  A-10  aircraft  was  flown.  The  environment 
included  mountaineous  terrain,  enemy  surface-to-air  missiles  (SAM)  and 
anti  aircraft  artillery  (AAA).  The  offensive  target  was  a  tank  located 
at  random  along  the  roadway.  The  pilots  flew  the  mission  in  real  time 
and  were  scored  on  offensive  and  defensive  procedures.  They  showed 
clear  learning  for  those  procedures  in  the  simulator. 

INTRODUCTION: 

The  purpose  of  this  study  was  to  test  the  feasibility  of  using  the 
Advanced  Simulator  for  Pilot  Training  (ASPT)  in  training  pilots  for  combat 
in  a  simulated  hostile  environment.  It  has  been  suggested  (Ref  1)  that  in 
the  future,  complex  flight  simulators  will  not  be  limited  simply  to 
procedures  and  instrument  training,  but  will  serve  the  much  more  important 
and  broader  need  of  training  combat  ready  pilots  in  the  complex  task  of 
tactics  development.  NASA's  experience  in  successfully  training  crews  to 
perform  moon  landings,  which  could  only  be  practiced  through  simulation,  has 
demonstrated  the  value  of  simulating  tasks  which  are  impractical  or 
impossible  to  practice  in  the  real  situation.  Experiences  of  the  Air  Force 
and  Navy  during  the  Viet  Nam  era  indicated  that  pilot  performance  in  a 
combat  situation  could  be  improved  considerably  by  providing  the  pilot  with 
experience  in  flying  mock  air-to-air  engagements  against  dissimilar  aircraft 
and  against  the  types  of  ground  defenses  he  would  be  coming  up  against  in 
combat  (Ref.  2).  This  type  of  experience  provided  the  pilot  with  the 
opportunity  to  develop  his  skills  at  offensive  and  defensive  strategies. 

The  success  of  this  program  led  directly  to  the  establishment  of  the  Red 
Flag  and  Blue  Flag  exercises  by  TAC  and  similar  training  exercises  by  other 
commands.  While  such  exercises  are  excellent  test  beds  for  new  tactics  on  a 
squadron  or  wing  level,  they  are  freqently  of  too  short  a  duration  for  the 
evolution  of  new  tactical  approaches  or  an  exploration  of  the  relative 
effectiveness  of  alternative  strategies  on  an  individual  pilot  level.  With 
the  use  of  a  high  fidelity  full  mission  simulator  such  as  the  ASPT,  it  is 
possible  to  expose  the  pilot  to  a  large  number  of  combat  simulations  of 
sufficient  fidelity  in  a  short  period  of  time  to  provide  him  an  optimum 
situation  for  offensive  and  defensive  skill  development.  Using  such 
simulation  will  engender  a  more  holistic  view  of  the  combat  mission  in  which 
the  pilot  will  be  required  to  meet  the  ever  changing  mission  requirements 
with  flexibility  and  judgment,  based  on  "experience".  The  present  study  is 
a  first  effort  to  demonstrate  that  the  simulator  can  be  used  in  this 
extended  role. 
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Combat  Environment 


Existing  visual  environments  on  the  ASPT  were  used  and  modified  to 
produce  the  computer  generated  Imagery  (CGI)  used  in  this  study.  The 
Imagery  was  presented  in  an  A-10  simulator  which  included  the  heads  up 
display  (HUD). 

A  map  of  the  basic  environment  can  be  seen  in  Figure  1.  It  consists  of 
an  area  of  about  10  miles  square.  Twelve  mountains  are  included  in  the 
area,  ranging  in  height  from  1400  to  6500  feet.  Also,  there  were  a  number 
of  small  rock-like  projections  between  the  mountain  ranges  on  the  right. 
Three  surface-to-air  missile  (SAM)  sites  were  located  in  the  central 
valley.  Each  SAM  site  had  the  following  capabilities: 

a.  Effective  slant  range  of  40,000  feet 

b.  Minimum  tracking  elevation  of  2.5° 

c.  Minimum  tracking  altitude  of  50  feet 

d.  Each  site  was  limited  to  six  firings  during  each  trial 

e.  Maximum  9-G  turn  capability 

f.  Acceleration  of  400  feet  per  sec2 

g.  Maximum  velocity  of  1600  feet  per  second 

The  SAM  site  closest  to  the  aircraft  was  the  one  activated.  If  the 
aircraft  was  outside  of  its  firing  envelope,  the  other  two  SAM  sites 
searched  to  see  if  the  aircraft  was  within  their  range.  When  the  aircraft 
was  within  range,  a  3.5  seconds  alternating  2025  Hz  spaced  tone  indicated 
radar  acquisition.  Inmediately  after  the  acquisition  warning,  a  three 
second  alternating  2025  Hz  tone  indicated  imminent  launch.  At  the  end  of 
the  tone,  the  SAM  fired  and  a  light  illuminated  to  indicate  launch.  The 
missile  could  be  seen  by  the  pilot  if  he  was  in  a  position  of  proper  viewing 
angle.  A  rapidly  alternating  tone  continued  for  as  long  as  the  SAM  was  in 
flight.  If  a  SAM  had  been  launched  and  the  aircraft  subsequently  exited  the 
firing  envelope  of  the  site,  the  SAM  would  abort.  If,  during  the  time  the 
SAM  waS  in  flight,  the  aircraft  entered  the  firing  envelope  of  another  SAM 
site,  the  SAM  in  flight  would  continue  its  course  unless  the  aircraft  exited 
the  envelope  of  the  launching  site,  in  which  case  the  SAM  aborted.  If  the 
aircraft  was  then  being  tracked  by  a  new  site,  a  new  acquisition  cycle  was 
Initiated.  If  hit  by  the  SAM,  the  aircraft  went  out  of  control  and  crashed. 

Co-located  at  each  SAM  site  was  an  anti-aircraft  artillery  battery 
(AAA),  consisting  of  four  23  inn  guns  as  well  as  an  isolated  AAA  site  to  the 
west  of  the  battle  area.  Each  AAA  site  had  the  following  capabilities: 

a.  Effective  slant  range  of  8,000  feet 

b.  Firing  rate  of  300  Rnd/Min 

c.  Minimum  tracking  altitude  of  50  feet 
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FIGURE  ] 


The  AAA  site  began  tracking  when  the  aircraft  flew  within  range.  If  the 
pilot  took  evasive  action,  which  consisted  of  +3  G's  or  more,  or  -1.5  G's  or 
less,  for  six  consecutive  seconds,  the  aircraft  would  evade  AAA  kill;  if 
not,  the  aircraft  would  be  hit  and  go  out  of  control.  Muzzle  flashes  were 
the  only  indication  that  the  aircraft  was  being  fired  upon.  The  aircraft 
did  not  have  the  capacity  of  destroying  the  SAM  or  AAA  batteries. 

The  enemy  target  in  this  environment  was  a  tank  which  was  located  at  one 
of  six  randomly  selected  positions  on  the  road  stretching  between  the  two 
mountain  ranges  at  the  Southern  end  of  the  battle  zone.  The  tank,  once 
placed  in  one  of  the  six  positions,  remained  stationary.  It  showed  muzzle 
flashes  from  its  cannon,  but  did  not  have  offensive  capabilities.  The  A-10 
aircraft  had  only  the  capability  of  firing  its  gun.  The  tank  was  considered 
destroyed  when  it  was  hit  by  one  round  during  the  first  sequence  of  firing, 
in  the  two-dimensional  space  occupied  by  the  tank. 

In  addition,  a  special  feature  was  added  to  the  environment  which 
allowed  the  pilot  to  see  his  own  shadow  when  his  altitude  was  below  500'  AGL. 

Subject  Population 


The  subjects  for  this  study  were  combat  ready  A-10  pilots,  as  determined 
by  Tactical  Air  Command  Headquarters.  They  were  all  in  Mission 
Qualification  Training  at  Davis-Monthan  AFB,  AZ.  They  ranged  in  age  from  26 
to  35  years.  Table  1  shows  a  breakdown  of  the  seven  subjects  with  respect 
to  flying  experience.  Three  of  them  had  combat  experience  and  all  but  two 
of  them  had  participated  in  Red  Flag  exercises. 

Experimental  Runs 

The  experimental  sequence  for  the  subjects  was  as  follows.  Each  subject 
was  briefed  individually  by  a  research  instructor  pilot  (IP)  on  the  combat 
environment,  including  all  the  offensive  and  defensive  characteristics.  He 
was  given  a  map  of  the  environment  which  he  was  allowed  to  study  before 
entering  the  simulator.  He  was  allowed  to  take  the  map  along  during  the 
simulator  run.  The  pilot  was  initialized  approximately  ten  miles  north  the 
forward  edge  of  the  battle  area  (FEBA)  in  IFR  conditions  at  an  altitude  of 
5500  feet.  The  pilot's  only  instructions  were  that  he  was  on  his  own  and 
that  his  mission  was  to  ingress  to  the  battle  area,  destroy  the  tank  and 
egress  safety  across  the  FEBA.  The  experimental  run  was  terminated  when  one 
of  the  following  occurred: 

a.  SAM  kill 

b.  AAA  kill 

c.  Terrain  crash 

d.  Over  G 

e.  Safely  crossed  FEBA 
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FLYING 

A- 10 

FIGHTER 

SUBJECT 

RANK 

AGE 

TIME 

TIME 

TIME 

COMBAT 

RED  FLAG 

#1 

CAPT 

28 

1100 

300 

900 

NONE 

X3  2-A7 ;  1 -A-10 

#2 

CAPT 

29 

1450 

225 

525 

NONE 

XI 

#3 

CAPT 

32 

2700 

300 

920 

677 

XI 

#4 

CAPT 

33 

2700 

50 

1000 

680 

X2  F-4 

#5 

MAJ 

35 

3700 

50 

2000 

350 

X3  F-105 

#6 

1LT 

26 

650 

70 

450 

NONE 

NO 

#7 

1LT 

26 

600 

65 

400 

NONE 

NO 

TABLE  ] 

H 

RAW 

M 

SCORES 

H 

M 

2P 

16 

15 

9 

11 

12 

8 

32 

am  pw 

H=  HIT  TANK 
M-  MISS  TANK 
S=  SURVIVED 
D=  DESTPOYED 
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As  soon  as  each  run  was  terminated,  the  pilot  was  immediately 
reinitialized  and  another  run  began.  Each  subject  flew  a  set  of  runs  in  the 
morning  and  a  second  set  in  the  afternoon.  Each  set  lasted  between  an  hour 
to  one  hour  and  a  half.  Each  subject  was  given  feedback  as  to  what  happened 
at  the  termination  point  of  each  run:  if  he  hit  the  tank,  if  he  survived, 
and  if  he  was  shot  down,  the  source  of  his  demise.  Between  periods,  the 
subjects  were  not  allowed  to  discuss  tactics  among  themselves  so  as  not  to 
bias  their  individual  approaches  to  solving  the  tactical  problems. 

Analysis 

On  the  basis  of  whether  the  pilot  hit  or  missed  the  target,  survived,  or 
was  destroyed,  a  chi  square  analysis  was  performed.  The  overall  analysis 
with  all  data  pooled  showed  statistical  significance  at  the  0.05  level. 

This  simply  means  that  the  pilot's  offensive  scores  (hitting  the  tank)  and 
defensive  procedures  (survival)  are  highly  correlated.  When  the  morning 
performance  was  compared  with  the  afternoon  performance,  there  was  a 
significant  increase  in  survival  and  offensive  capabilities  in  the 
afternoon.  Table  2  shows  a  comparison  of  A.M.  vs  P.M.  scores. 


A  scoring  system  was  developed  in  order  to  plot  curves  which  were 
representative  of  individual  performance  with  respect  to  offensive  and 
defensive  outcomes.  Scores  assigned  were  as  follows: 


Defensive: 

+2  No  detection 

+1  Detected/not  killed 

0  Killed  by  SAM,  AAA,  Crash,  or  over  "G" 

Offensive: 

+2  Fired/kill  tank 

+1  Fired/missed  tank 

0  No  fire  on  tank 


The  combined  (offensive  plus  defensive)  scores  of  all  subjects  indicated 
an  overall  improvement  after  repeated  training  sessions.  The  mean  total 
scores  for  the  seven  subjects  across  the  20  training  trials  are  shown  in 
Figure  2.  The  scores  shown  with  a  possible  range  of  0  to  4  indicate  the 
combined  offensive  and  defensive  capability  of  the  pilot. 

An  examination  of  the  data  from  a  separate  consideration  of  offensive 
and  defensive  scores,  shown  in  figure  3,  provides  some  further  insight  into 
the  apparent  decline  of  performance  trials  11  and  18.  Scores  for  both 
offensive  and  defensive  performance  improved  rapidly  from  trials  1-8,  with 
offensive  performance  scores  entering  the  acceptable  range  (greater  than 
+1.0)  from  trials  5/6  on.  The  marked  jump  in  average  offensive  scores  from 
trials  7/8  to  trials  9/10  indicate  that,  in  general,  the  pilots  had  "solved" 
the  offensive  portion  of  the  problem  confronting  them.  However,  at  no  time 
during  this  period  (trials  1-10)  did  the  average  defensive  score  move  into 
the  acceptable  range  (greater  than  1).  That  is,  on  the  majority  of  the 
trials  the  average  pilot  was  being  detected  and/or  shot  down  -  although  by 
trials  9/10  most  pilots  were  getting  the  target  tank  first.  This  is 
particularly  significant  in  view  of  the  fact  that  the  scenario  was 
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specifically  set  up  so  that  the  pilot  could,  in  fact,  acquire  and  destroy 
the  tank  and  return  without  being  detected  or  shot  at  by  the  enemy  AAA  or 
SAM  batteries.  Clearly,  the  pilots  had  not  "solved"  the  defensive  aspects 
of  the  problem  by  the  end  of  the  first  ten  sessions.  The  nature  of  the  data 
in  figure  3  in  conjunction  with  that  in  table  2  suggests  that  the  pilots 
began  to  attend  more  to  the  defensive  aspects  of  the  problem  scenario  in  the 
afternoon  sessions  (trials  11-20). 

The  data  clearly  indicate  that  the  pilots  overall  have  learned  to 
perform  better  offensive  and  defensive  maneuvers  after  repeated  trials  in 
the  simulator.  However,  the  offensive  problems  are  apparently  solved  much 
more  rapidly  than  the  defensive  ones.  Possibly  because  they  are  the  most 
salient  aspects  of  the  problem  and  due  to  the  nature  of  the  pilots'  previous 
training.  The  defensive  problems  are  solved  much  more  slowly  and  require 
greater  practice. 

Conclusions 


This  being  the  first  attempt  to  use  the  ASPT  (or  any  other  simulator) 
for  the  production  of  a  full  combat  scenario,  the  preliminary  results  are 
quite  encouraging.  The  visual  system  was  programmed  to  almost  full  capacity 
to  produce  the  battle  environment  in  which  the  pilot  was  free  to  fly  and 
develop  his  own  tactics  in  destroying  the  target  and  surviving  the  threat 
environment.  In  this  sense,  the  pilot  is  operating  in  a  completely  open 
loop  fashion,  as  he  would  largely  operate  in  true  combat,  and  as  such,  may 
be  able  to  develop  the  flexibility  and  problem  solving  strategies  he  will  be 
called  upon  to  develop  in  actual  combat.  The  learning  curves  indicate  the 
pilots  are  ck  tiring  and  integrating  appropriate  information  so  as  to 
improve  their  tactical  performance. 

The  A- 10  pilots'  acceptance  of  this  type  simulation  was  remarkably 
positive.  All  of  the  combat  ready  pilots  appeared  very  enthusiastic  about 
the  utility  of  this  sort  of  simulation.  They  pointed  out  that  they  had 
never  had  an  opportunity,  except  for  real  combat,  to  deal  with  the  combat 
environment  in  a  holistic  way  as  it  was  presented  to  them  in  the  simulator. 
The  demand  of  dealing  with  the  SAM  and  AAA  weapons  while  conducting  their 
offensive  maneuvers,  they  felt,  was  particularly  instructive.  The  "pop-up" 
maneuver,  for  example,  which  is  part  of  their  standard  repertoire,  has  to  be 
clearly  modified  when  in  range  of  either  the  SAM  or  AAA  site.  They  were  in 
unanimous  agreement  that  the  simulation  was  very  compelling  and  believable. 
They  noted  that  with  the  use  of  wide  screen  visual  presentation,  in  conjunc¬ 
tion  with  the  G-seat  and  G-suit,  the  use  of  platform  motion  was  not 
necessary  to  provide  a  highly  realistic  training  environment. 

In  short,  with  the  limited  combined  scenario  presented,  combat  ready 
pilots  exhibit: 

a.  Clear  learning  of  offensive  and  defensive  tactics. 

b.  Favorable  responses  to  such  training 
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Recommendations 

It  appears  that  the  simulator  may  be  able  to  supply  an  important 
training  methodology  for  tactics  development.  With  the  development  of 
greater  computer  capacity,  the  visual  and  other  systems  may  be  capable  of 
producing  dynamic  changing  combat  environments  which  would  allow  the  combat 
ready  pilot  to  maintain  his  combat  skills  at  a  very  high  level.  Flight 
simulation  should  fill  the  training  void  of  those  flying  tasks  which  are 
impractical  or  impossible  to  practice  in  the  real  aircraft.  It  is, 
therefore  recommended  that  continued  development  of  combat  environments  be 
made  with  ever  increasing  variability  and  flexibility  so  the  total  concept 
of  simulator  tactics  development  can  be  more  thoroughly  tested  and 
exercised.  It  is  also  recommended  as  a  follow-on  from  such  development, 
that  comparisons  be  made  between  the  performance  scores  in  exercises  such  as 
Red  Flag  with  pilots  trained  in  simulation  and  with  pilots  who  have  not  had 
such  exposure.  Such  a  program  could  very  well  determine  what  sort  of 
transfer  of  training  is  taking  place  from  the  simulator  to  the  tactical 
exercise  field. 
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EFFECTS  OF  VISUAL  AND  MOTION  CUES 


ON  PILOT  EFFECTIVENESS  DURING  ENGINE-OUT  TRAINING 


ABSTRACT 


This  study  was  designed  to  determine  the  need  for  visual  and/or  motion 
systems  during  engine  out  training  in  non-centerline  thrust  transport 
category  aircraft.  The  KC-135A  Flight  Station  portion  of  Boeing's  Weapon 
System  Trainer  (WST)  with  a  six  degree-of -freedom  motion  system  and  three 
window  daylight,  dusk,  and  night  visual  system  was  used  for  this  study. 
Qualified  KC-135A  air  crews  with  900  to  3000  hours  experience  were  randomly 
presented  normal  and  engine-out  takeoffs  with  motion/no  motion  and  varied 
visual  conditions.  Pilot  Performance  Data  was  recorded  by  the  automatic 
data  extraction  system  of  the  WST  and  six-channel  strip  chart  recorder. 
Pilot  recognition  and  response  time  was  chosen  for  this  analysis  as  the  key 
to  good  performance  during  engine  failure. 

Fastest  response  time  (and  correspondingly  lowest  sideslip,  bank  angles, 
and  roll  rates)  occurred  in  cue  configurations  involving  motion  and  out-of¬ 
window  visual  displays.  This  conf iguration  also  produced  the  most 
efficient  pilot  management  of  engine  loss.  Subjective  data  suggested  that 
although  the  subjects  were  committed  to  the  need  for  motion  cues  in  flight 
training  devices,  their  error  rates  were  very  high  when  perceiving  and 
describing  actual  motion  cues. 

INTRODUCTION 


A  major  advantage  of  flight  training  devices  is  their  ability  to  train 
pilots  in  situations  which  would  be  unsafe  in  true  aircraft.  Engine 
failure  during  takeoff  is  one  such  example.  Here,  it  is  critical  for  a 
pilot  to  recognize  the  emergency  and  initiate  corrective  action. 

A  multiple-engine,  non-centerline  thrust  aircraft  such  as  the  KC-135  reacts 
to  engine  failure  with  an  immediate  yaw  in  the  direction  of  the  failed 
engine  and  sideslip  in  the  opposite  direction.  If  the  emergency  occurs 
during  or  after  liftoff  the  additional  problems  of  "Dutch  Roll"  must  be 
countered.  The  frequency  of  the  Dutch  Roll  is  such  that  if  a  pilot  takes 
two  to  three  seconds  to  react  to  the  engine  loss  he  will  excite  or 
strengthen  the  roll  and  yaw  tendencies.  The  KC-135A  Flight  Manual  states 
that  a  "pilot  reaction  time  of  3  to  4  seconds  may  allow  a  yaw  angle  of  9  to 
12  degrees  and  bank  angles  of  up  to  15  degrees  to  develop  prior  to  the 
initiation  of  corrective  action".  Further,  the  flight  manual  states  in  a 
WARNING  that  the  pilot  should  "use  extreme  caution  when  bank  angle  is  used, 
since  the  outboard  engine  pod  will  make  contact  at  a  bank  angle  of 
approximately  8  degrees"  (Ref.  10).  At  the  point  of  takeoff  the  outboard 
pods  are  6.3  feet  above  the  runway.  Reaction  time  is  thus  critical  to  good 
performance  during  engine  failure.  The  pilot  who  has  had  the  opportunity 
to  practice  the  engine  failure  emergency  In  a  realistic  training  device 
should  manage  such  an  emergency  better  than  one  who  encounters  engine  loss 
for  the  first  time  during  the  actual  event. 
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The  six  degree-of -freedom  flight  training  device  with  visual  displays 
offers  the  opportunity  to  address  nagging  questions  as  to  the  comparative 
contributions  of  visual  and  vestibular  motion  cues  in  training  tasks. 
Arguments  for  and  against  motion  and  visual  system  are  familiar  to 
researchers  in  training  devices.  (Caro,  Ref.  1;  Cohen,  Ref.  2;  Gundry, 

Ref.  4;  Koonce,  Ref.  5;  Schwing,  Ref.  8). 

It  can  be  argued  that  the  dynamics  of  aircraft  responses  to  sudden  power 
asymmetry  are  first  detected  by  the  vestibular  receptors,  with  visual 
stimulus  sources  used  as  consultant  identifiers  or  as  confirmers  of  senses 
felt  (Cohen,  Ref.  2).  Alternatively,  one  can  argue  that  body  action  occurs 
first  in  the  case  of  learned  reactions  when  the  source  of  disturbance  is 
previewed  with  the  eyes  as  well  as  from  a  motion  cue.  This  argument  relies 
on  the  fact  that  the  human  vestibular  apparatus  tends  to  summate  the 
various  vectors  of  acceleration  operating  (e.g.  yaw,  sideslip,  roll,  pitch) 
and  report  them  as  a  mean  vector  of  motion,  possibly  contributing  to 
vertigo  (McCormick,  Ref.  6).  Thus,  motion  cues  at  times  present  a  poor 
basis  for  corrective  action  during  emergencies.  On  the  other  hand  visual 
cues  are  useful  only  if  the  user  happens  to  be  looking  in  the  area  in  which 
the  cue  is  provided.  In  addition  to  representing  the  visual  world  a  visual 
system  can  have  a  motion  effect  as  investigated  by  Young,  who  states  that  a 
visual  field  can  "trick"  the  human  motion  sensing  apparatus  into  perceiving 
motion  when  none  is  occurring  (Young,  Ref.  12).  This  phenomenon  is  most 
evident  in  360°  or  wide  screen  movie  presentations  such  as  those  at 
Disneyland. 

It  appears  that  the  issue  of  the  relative  importance  of  motion  and  visual 
cues  will  not  easily  be  resolved  and  that  cost  may  become  the  deciding 
factor  in  the  purchase  of  visual  or  motion  systems  for  training  devices. 

This  study  was  conducted  to  determine  the  value  of  visual  and  motion  cues 
on  training  effectiveness.  From  an  informal  study  on  motion 
requirements/needs,  an  emergency  situation  -  outboard  engine  failure  at 
takeoff  -  was  chosen  as  a  standard  case.  This  allowed  the  study  team  to 
concentrate  on  the  aspects  of  visual  and  motion  cues  contributing  to  pilot 
effectiveness  in  engine-out  training. 

METHODS 

Subjects.  Fifteen  KC-135A  aircraft  commanders  volunteered  as  subjects. 
Pilot  experience  level  ranged  from  900  flight  hours  to  3000  hours,  with  an 
average  of  2,100  hours.  Minimum  flight  time  in  the  KC-135A  was  500  hours; 
mean  KC  experience,  1,600  hours. 

Equipment.  The  KC-135A  Flight  Station  portion  of  Boeing's  Weapon  System 
Trainer  (WST)  was  used  for  the  study.  This  training  device  has  a  six 
degree-of -freedom  motion  system  (Figures  1  and  2).  The  three  window  visual 
system  has  the  capability  of  providing  daylight,  dusk,  and  night  visual 
scenes  projected  onto  the  pilot's  and  copilot's  front  windows  and  pilot's 
side  window. 


Data  Collection.  Data  was  collected  using  three  different  and  Independent 
methods.  The  primary  source  of  data  came  from  the  WST  Automatic  Data 
Extraction  System  (ADE).  This  system,  which  stores  data  in  disc  files,  is 
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Figure  2  KC-135A  WST  with  a  number  4  engine  failure  (extreme  case) 


used  by  the  training  instructor  for  post-mission  scoring  and  standardized 
feedback  to  the  student.  Parameters  recorded  include  such  information  as 
variation  from  centerline,  heading,  yaw  rate  and  acceleration,  control  column 
pitch,  and  bank  angle.  Secondary  performance  parameters  were  recorded  on  a 
six-channel  strip  chart  recorder  which  graphically  illustrated  engine 
failure,  rudder  pedal  angle,  bank  angle,  roll  rate,  sideslip,  and  minimum 
engine  pod  clearance.  Finally,  each  pilot  was  asked  to  evaluate  the  motion 
and  visual  condition  present  during  the  takeoffs  as  to  the  adequacy  of  each 
for  representing  engine  failure  and  normal  takeoffs. 

Procedures  and  Design.  Study  periods  lasted  two  and  one-half  hours.  Prior 
to  beginning  each  session,  subject  pilots  were  given  a  tour  of  the  WST 
facilities.  Motion  system  safety  briefings  and  an  overview  of  the  study 
followed  the  tour. 

A  pilot  completed  his  takeoff  runs  in  random  order  of  motion/no  motion, 
normal  or  engine  failure,  with  the  same  visual  scene.  Prior  to  beginning 
data  gathering,  each  pilot  was  given  a  familiarization  takeoff  and  landing 
using  the  experimental  conditions. 

Outboard  engines  One  (pilot's  far  left)  and  Four  (copilot's  far  right)  were 
chosen  as  failure  candidates  because  of  the  adverse  yaw  caused  by  their 
sudden  thrust  loss.  Thrust  loss  occurred  instantaneously,  simulating  a 
catastrophic  failure  of  that  power  source.  All  failures  began  when  the 
aircraft  reach  6  degrees  positive  pitch;  in  this  attitude  the  aircraft 
nosewheel  is  off  the  runway,  a  positive  flight  path  has  been  established, 
full  main  gear  extended,  but  lift-off  has  not  occurred.  The  experiment  was 
planned  as  a  Split-Plot  factorial  design  with  nine  different  motion  cues 
(eight  combinations  of  six  degrees-of-freedom  plus  pitch  and  one  no-motion 
condition);  three  levels  of  out-of -cockpit  visual  display  (no  display, 
forward  display,  and  forward  plus  left  peripheral  display);  and  three 
conditions  of  engine  failure  (left  outboard,  right  outboard,  and  no  engine 
failure).  Constraints  imposed  by  equipment  availability  left  many  cells  in 
the  design  with  inadequate  entries.  This  limits  the  elegance  of  statistical 
analysis  at  present  but  allows  preliminary  results  to  be  presented. 


EXPERIMENT Al  DESIGN 
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Alternatively,  a  small  n  approach  was  adopted  for  identifying  effects  of 
experimental  variables  and  suggesting  trends  (Robinson  and  Foster,  Ref.  7). 
This  analysis,  which  must  be  considered  tentative,  utilized  pooled  motion  cue 
combinations  rather  than  separate  analysis  of  the  eight  individual  motion 
cues  and  interactions.  Moreover,  the  dependent  measure  to  be  reported  is 
pilot  response  time  (elapsed  time  from  engine  failure  to  compensatory  control 
input),  which  correlates  positively  with  other  dependent  measures  (bank 
angle,  engine  pod-to-ground  clearance,  yaw  rate  and  angle,  and  sideslip  angle 
(Example  Figure  3).  Means  are  reported  due  to  their  ease  in  graphing  data 
and  difficulty  of  presenting  such  large  numbers  of  raw  scores. 


FIGURE  3  RESPONSE  T l  ME  »s.  BANK  ANGLE 


The  pilots  who  participated  were  all  highly-trained,  but  had  virtually  no 
experience  in  simulators  with  motion  systems.  To  both  introduce  the  pilots 
to  motion  cues  and  assure  them  that  the  motion  system  was  operable,  we  made 
no  effort  to  damp  or  conceal  the  simulator's  vertical  excursions  as  it  came 
up  from  rest  or  returned  to  rest.  All  test  sessions  began  and  ended  with 
such  an  excursion;  but  on  no-motion  runs,  the  motion  system  was  "clamped" 
off  with  the  simulator  In  Its  raised  "neutral"  position,  and  no  further 
motion  cues  were  Imparted.  These  preparatory  simulator  vertical  excursions 
occurred  at  the  beginning  of  each  session,  after  twelve  takeoffs  when  a 
stretch  break  was  taken,  and  again  after  the  24th  takeoff  run.  To  alleviate 
boredom,  after  every  fifth  run,  pilots  were  given  a  chance  to  fly  an 
Instrument  approach  and  landing. 
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Following  each  takeoff  run,  pilots  were  asked  to  evaluate  the  adequacy  of 
both  motion  and  visual  cueing  systems  for  engine-out  training.  Questions 
about  motion  cues  were  asked  following  no-motion  runs  as  well,  to  clarify 
whether  the  subjects  could  discriminate  motion  from  no-motion  conditions  in 
the  absence  of  verbal  prompts  and  feedback  from  the  experimenter.  Of  six 
possible  motion  cues  imparted  on  motion  runs,  the  subject  had  to  identify  at 
least  two  of  the  three  primary  motion  axes  (pitch,  roll,  and  yaw)  plus 
either  the  third  primary  axis  or  one  of  the  secondary  axes  (surge,  heave  and 
sideslip)  in  order  to  be  credited  with  correct  motion  cue  identification. 

RESULTS 

Despite  limitations  in  our  data  due  to  limited  facility  time,  a  case  can  be 
made  for  the  reliability  of  the  data  by  comparing  pilot  compensatory 
response  to  engine  failure  with  and  without  motion  cues  where  forward  only 
vision  was  provided.  Here,  eight  of  sixteen  subjects  contributed  data  (167 
takeoffs).  Mean  response  time  for  the  no-motion  condition,  forward  vision 
only,  was  1.71  seconds,  which  corresponds  to  the  grand  mean  for  all  subjects 
for  all  takeoffs  (1.83  seconds).  With  motion,  these  sana  subjects  responded 
slightly  quicker  (mean  =  1.56  seconds),  a  difference  which  is  negligible 
(variance  for  all  runs  for  all  subjects  =  0.22  seconds,  standard  deviation  = 
0.47  seconds). 


REACTION 
TIME  IN 
SECONDS 


WINOOW  WINDOW 


NO  MOTION  MPT  I  Q|| 

FIGURE  «  .  MEAN  LATENCIES  IN  SECONDS  BETWEEN  ENGINE  LOSS  ANO  PILOT- INITIATED  CORRECTIVE  ACTION 


52 


Other  comparisons  in  the  data  must  be  considered  less  reliable,  due  to  fewer 
data  points,  but  provide  interesting  results.  For  example,  longest  pilot 
response  times  are  associated  with  absence  of  vision  cues,  regardless  of 
presence  of  motion  cues  (2.19  seconds  with  motion  vs  2.78  seconds  without 
motion).  Addition  of  a  left  peripheral  window  display  to  the  forward 
display  quickens  pilot  response.  (No  motion  -  1.71  seconds,  forward  visual 
vs  1.60  seconds,  forward-plus -side  window.  With  motion  -1.56  seconds, 
forward  vs  1.39  seconds,  forward-plus-side  window  display.) 

A  'isparity  in  pilot  management  of  engine  loss  was  apparent  in  all  visual- 
ated  data.  Response  to  engine  #1  (far  left)  was  consistently  more 
efficient  than  to  engine  #4.  (Example:  Forward-plus-pilot's  left  window, 
no  motion,  1.40  —  1.72  sec.,  respectively;  with  motion,  1.08  —  1.60  sec., 
respectively).  This  implies  a  contribution  to  engine  loss  detection  by  the 
presence  of  peripheral  cues  in  the  event  of  left  outboard  engine  failure 
whenever  the  pilot  is  looking  out  forward  as  the  primary  visual  source.  The 
data  implies  suppor  for  changes  in  aircraft  cockpit  design,  and  for  the 
inclusion  of  side  window  visuals  on  training  devices.  Visual  confirmation  of 

engine-out  conditions  can  occur  and  further  quicken  positive  action  if  the 
pilot  sees  the  failed  engine's  instruments.  Without  an  out-of -window 
scene,  time  required  to  detect  engine  #1  or  #4  failure  is  virtually  the 
same  (2.72  seconds,  engine  #4,  —  2.82  seconds,  engine  #1).  Figure  5  is  a 
representation  of  the  KC-135  engine  instrument  panel.  Further  study  needs 
to  be  undertaken  to  determine  if  the  copilot  has  a  similar  problem  with 
engine  #1.  More  work  with  side  window  visuals  should  be  done  to  prove  that 
the  side  window  displays  contribute  to  increased  training  efficiency. 


TABLE  2. 

PILOT  RESPONSE  TIMES  (SECONDS)  TO  LOSS  OF  #1  AND 
#4  ENGINES  WITH  VARIED  VISUAL  AND  MOTION  CUES. 


NO  ACTIVE 
WINDOWS 

FORWARD  WINDOWS 
ACTIVE 

FORWARD  AND  LEFT 
SIDE  WINDOWS 

AFFECTED 

ENGINE 

NO 

MOTION 

MOTION 

MOTION 

MOTION 

MOTION 

MOTION 

MEANS 

#1 

2.82 

1.84 

1.57 

1.52 

1.42 

1.08 

1.71 

M 

2.72 

2.29 

1.86 

1.64 

1.78 

1.60 

1.98 
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Specific  Questions  About  Motion  Cues.  When  neither  motion  nor  visual  cues 
were  operating,  judgments  by  pilots  that  "motion"  cues  were  adequate 
occurred  on  16%  of  takeoff  runs  (Figure  6,  Line  1).  Moreover,  they 
erroneously  specified  the  presence  of  motion  cues  on  84%  of  these  minimal 
cue  runs  (Line  7).  When  visual  cues  were  present  (Figure  6,  Lines  2  and 
3),  subjects'  ratings  of  the  adequacy  of  motion  cues  increased  dramatically 
(79,  80%)  while  the  level  of  confusion  about  the  presence  of  motion 
increased  to  93%  (Line  8).  In  summary  pilots  were  apparently  convinced  by 
the  window  displays  that  motion  cues  were  operating  when  none  were,  and 
these  "cues"  were  regarded  as  adequate  for  management  of  an  engine  loss 
emergency. 
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figure  6.  RiLCr  questionnaire  results 


The  addition  of  motion  cues  (Lines  4,  5,  and  6)  led  to  judgments  of 
increased  adequacy  but  did  not  clear  the  perceptual  confusion  regarding  the 
presence  or  nature  of  motion  cues  (Lines  10,  11,  and  12).  Pilots  tended  to 
be  confused  about  the  presence  of  motion  or  to  deny  in  some  cases  that  it 
was  there.  The  most  accurate  identification  of  motion  cues  occurred  on 
runs  where  motion  and  full  visual  cues  were  present  (Line  12),  but  even 
here,  26%  of  the  time  incorrect  judgments  occurred. 
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An  answer  to  the  errors  in  motion  perception  caused  by  the  inclusion  of 
visual  systems  may  come  from  the  phenomenon  of  "self  motion"  as  explained 
by  Young  (Ref.  12).  Young  presents  data  that  suggests  peripheral  cues  are 
required  for  self  motion  to  occur  and  that  subjects'  mental  set  is  very 
important.  In  the  present  case,  the  subject  was  "set"  to  perceive  motion 
by  the  elaborate  tour  of  the  facilities  and  preparatory  motion  excursions 
described  earlier  in  this  paper.  However,  the  peripheral  cues  provided  in 
out-of -window  forward  visuals  are  minimal;  and,  thus,  should  not  have 
encouraged  the  phenomenon  of  self  motion.  This  fact  may  have  been  offset 
by  the  acceleration  or  change  of  visual  field  velocity  during  takeoff  which 
can  induce  the  sensations  of  pitch  and  roll  so  long  as  the  velocity  is, 
according  to  Young,  "in  order  of  the  threshold  of  the  semicircular  canals 
(approximately  0.2°/sec.  j  and  of  the  otoliths  (.005  g's)." 

Self  motion,  cannot  account  for  the  other  side  of  the  coin  when  motion  is 
added  to  the  visual  presentation.  In  this  case,  false  sensations  of  the 
body  at  rest  must  have  been  present.  In  summary,  the  subjective  data  imply 
that  our  pilots  perceived  motion  when  none  was  occurring  and  yet  sometimes 
failed  to  perceive  real  simulator  motion. 


General  Ratings  of  Visual/Motion  System  Contributions.  In  addition  to  post¬ 
takeoff  interrogation,  pilots  were  given  general  questionnaires  about  visual 
and  motion  systems  in  trainers  at  the  conclusion  of  the  study  sessions.  A 
summary  of  subjects'  attitudes  about  visual/motion  systems  follows 
(percentage  indicates  agreement  or  strong  agreement  with  the  statement): 

o  Is  a  motion  system  essential  in  an  aircraft  simulator?  (70%) 

o  I  enjoy  attending  a  simulator  with  motion  more  than  one  without. 

(100%) 

o  A  disturbance  motion  system  is  the  most  important  factor  in 
recognizing  aircraft  problems  or  emergencies.  (80%) 

o  A  visual  system  is  essential  for  training  devices.  (90%) 

o  I  enjoy  attending  a  simulator  with  a  visual  system  more  than  one 
without.  (100%) 

o  A  motion  system  is  not  needed  with  expanded  visuals.  (30%  agreed 
with  this  statement,  30%  disagreed,  and  the  remainder  were 
undecided) . 

These  attitude  data,  though  tentative,  suggest  that  pilots  consider  motion 
and  visual  systems  about  equally  important,  enjoy  simulators  with  both,  and 
are  unsure  whether  visual  cues  can  adequately  supplant  motion  cues.  They  are 
more  sure  that  motion  cues  are  essential  in  aircraft  training  devices  and 
that  motion  cues  are  most  important  in  recognizing  emergencies.  The  latter 
is  in  contrast  to  the  pilots  demonstrated  inability  to  determine  whether  the 
motion  system  was  on  or  off. 
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CONCLUSIONS 


Pilot  Performance.  An  intriguing  finding  in  the  present  study  was  that 
pilots  consistently  responded  more  rapidly  to  loss  of  #1  engine  (left 
outboard)  than  to  loss  of  #4  (right  outboard).  Our  considered  opinion  is 
that  instruments  associated  with  #1,  being  peripherally  perceptible  to  the 
pilot,  provide  early  cues  to  engine  failure. 

In  the  present  study  window  displays  led  to  improved  pilot  performance 
despite  the  disadvantaged  visual  presentation,  where  the  horizon  dropped  out 
of  view  at  the  moment  of  engine  failure.  Improved  performance  with  visual 
cues  occurred  in  both  the  motion  and  no-motion  conditions.  The  combination 
of  forward  plus  left  side  vision  and  no-motion  produced  pilot  reactions  as 
timely  as  the  combination  of  motion  cues  plus  a  forward-only  window  scene. 
These  data  conflict  with  those  of  DeBerg,  McFarland,  and  Showalter  whose 
subjects  failed  to  show  an  advantage  for  visual  displays  in  flight,  (Ref. 

3).  In  general,  pilots  performed  better  with  an  active  visual  system  and  an 
inactive  motion  system  than  they  did  when  only  the  motion  system  was  active. 

Subjective  Responses.  The  pilots  reported  strong  beliefs  in  the  importance 
of  motion  cues  in  managing  engine  loss  and  stated  that  disturbance  motion 
cues  are  most  important  in  that  context.  They  also  manifested  a  strong 
preference  for  simulators  with  visual  and  motion  systems.  Still,  their  post¬ 
trial  attempts  to  identify  the  particular  cues  just  experienced  were 
characterized  by  excessive  errors.  The  most  dramatic  of  these  occurred  with 
both  visual  and  motion  systems  active.  Some  pilots  denied  that  motion  had 
actually  occurred  (8%  of  the  judgments).  The  opposite  effect,  where  pilots 
reported  the  presence  of  motion  when  none  had  occurred,  was  substantially 
more  frequent  (93%)  and  suggests  that  a  phenomenon  such  as  visually-induced 
"motion"  was  operating. 

Future  Research.  A  full  study  of  pilot  performance  based  on  the  preliminary 
research  outlined  above  should  be  undertaken.  The  failure  management 
disparity  between  Engines  One  and  Four  needs  to  be  resolved  and  possible 
action  taken  to  improve  instrument  configuration  so  that  response  latencies 
are  not  degraded  by  cockpit  instrument  positioning. 

The  effect  of  side  windows,  i.e.  peripheral  cues,  on  performance  should  be 
further  assessed  and  the  training  efficiency  of  this  configuration  explored. 
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ABSTRACT 

This  paper  describes  next  generation  CIG  system  that  produces  a  more 
faithful  representation  of  the  real  world  than  current  edge  systems.  Using 
textured  curved  surfaces  to  model  scene  features,  the  system  produces 
realistic  scene  detail  providing  better  flying  cues  in  training  simulators. 
Advanced  techniques  for  scene  modeling  and  imaging  are  implemented  in  a  unique 
system  architecture  to  achieve  real-time  image  generation  of  superior  quality. 

INTRODUCTION 

As  computer  image  generation  (CIG)  has  developed  over  the  past  two 
decades,  the  simulator  community  has  come  to  rely  on  it  as  the  best  means  of 
providing  visual  cues  for  training.  To  achieve  the  high  data  rates  required 
by  real-time  displays,  early  CIG  systems  had  to  simplify  the  computation  by 
reducing  the  world  they  modeled  to  a  linear  approximation  defined  by  edges. 
Although  edge  systems  have  been  greatly  refined  over  the  years,  the  basic 
limitations  of  the  approach  persist.  Consequently,  current  CIG  systems  still 
produce  a  stylized  representation  of  scene  features  that  is  characterized  more 
by  linear  artifacts  than  by  the  true  nature  of  the  re^l  world.  In  addition, 
current  CIG  systems  are  unable  to  provide  adequate  scene  detail.  As  a  result 
of  these  shortcomings,  current  CIG  systems  are  severely  limited  in  the  quality 
of  training  that  they  can  produce. 

In  an  attempt  to  satisfy  the  stringent  requirements  of  effective 
training  current  CIG  manufacturers  are  striving  to  increase  scene  content  by 
increasing  the  edge  capacity  of  their  systems.  But  serious  cost  barriers 
stand  in  the  way.  As  the  edge  count  goes  up  so  do  the  costs  of  scene 
modeling,  data  management  and  image  generation.  CIG  manufacturers  are  relying 
on  a  breakthrough  in  hardware  or  edge-handling  techniques  to  surmount  these 
barriers,  but  they  ignore  the  oasic  problem.  The  simplistic  straight  edge  is 
an  inefficient  and  ineffective  primitive  with  which  to  model  the  subtle, 
non-linear  complexity  of  the  real  world. 

To  avoid  the  limitations  inherent  in  edge  systems,  the  Grumman  Aerospace 
Corporation's  Advanced  Computer  Image  Generation  project  (ACIG)  is  pursuing  a 
unique  approach  to  develop  a  next-generation  CIG  system.  To  model  a  scene  we 
use  a  geometric  data  base  of  curved  surfaces  to  represent  the  non-linearity  of 
the  real  world.  A  mathematically  generated  texturing,  mapped  onto  the 
surfaces,  produces  essential  scene  detail  unavailable  in  current  systems.  The 
total  data  base,  composed  of  parameters  of  analytic  functions  Instead  of 
discrete  vertex  points,  is  much  more  compact  than  that  required  for  edges,  yet 
the  scene  content  is  far  greater  and  represents  the  real  world  much  more 
faithfully.  Our  goal  is  to  develop  techniques  that  take  advantage  of  this 
streamlined  data  base  to  reduce  the  costs  involved  in  scene  modeling,  data 
storage  and  transfer,  and  image  generation,  and  to  provide  the  essential 
flying  cues  required  to  assure  effective  training.  Because  we  realize  that 
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this  effort  will  be  meaningful  only  in  its  ultimate  application,  we  have 
included  as  a  major  part  of  our  project  the  development  of  a  novel  system 
architecture  to  implement  our  CIG  techniques  in  cost-effective  real-time 
hardware. 

This  paper  will  describe  the  research  and  development  work  performed  in 
the  ACIG  project  and  work  funded  by  the  Air  Force  under  the  contract  entitled 
Advanced  Computer  Image  Generation  Visual/  Sensor  Simulation  (AVSS). 

PROGRESS  ON  ACIG 

In  order  to  avoid  the  limitations  of  edges  in  modeling  the  natural 
curvature  of  real-world  features,  we  have  chosen  a  scene  data  base  that 
includes  curved  as  well  as  planar  surfaces.  The  simplest  form  of  curved 
surface  is  the  quadric,  or  second-order,  surface  (1).  Although  quadric 
surfaces  have  been  studied  extensively  for  non-real -time  image  generation 
applications,  such  as  computer-aided  design,  they  have  been  considered  only 
for  limited  use  in  real-time  CIG  systems  (2).  This  neglect  of  a  very  powerful 
modeling  tool  stems  partly  from  the  early  success  of  edge  systems  and  partly 
from  real  geometric  limitations  of  the  quadric  surface. 

Because  the  quadric  surface  is  the  simplest  curved  surface  it  is  not  as 
easily  fitted  to  irregular  terrain  data  as  are  higher-order  surfaces.  For 
this  reason,  more  attention  has  been  given  to  higher-order  surfaces,  in 
particular  the  bicubic  surface.  However,  even  though  bicubic  surfaces  can 
model  irregular  terrain  more  easily  and  with  greater  slope  continuity  than 
quadrics  can,  they  require  prohibitive  computation  to  determine  silhouette 
boundaries,  to  produce  intensity  shading  and  texturing  and  to  implement 
antialiasing  (3).  In  addition,  bicubic  surfaces  are  best  implemented  as  sets 
of  surface  patches.  The  number  of  patches  involved  and  the  complexity  of  the 
patch  definition  produce  a  cumbersome  and  costly  data  base.  Modeling  with 
bicubic  patches,  therefore,  tends  to  be  another,  more  sophisticated,  brute 
force  approach  whose  success  depends  on  major  breakthroughs  in  both  software 
and  hardware. 

Our  strategy  has  thus  been  to  develop  techniques  that  take  advantage  of 
the  analytic  simplicity  and  minimize  the  limitations  of  quadric  surfaces  to 
produce  the  most  cost-effective  visual  display  for  simulation. 

Implementation  of  Quadric  Surfaces 

Quadric  surfaces  can  model  a  scene  as  the  eye  perceives  it,  as  a  set  of 
individual  features  each  defined  by  a  small  number  of  surfaces.  We  can  model 
a  significant  scene  feature  such  as  a  tree  or  a  hill  with  one  or  two  surfaces 
as  opposed  to  the  numerous  planar  patches  required  in  edge  systems.  Although 
a  great  deal  of  flexibility  in  modeling  complex  features  can  be  gained  by 
using  multiple  intersecting  quadric  surfaces,  this  is  not  cost-effective. 
Surfaces  of  second  order  intersect  to  produce  curves  of  fourth  order  which 
require  too  much  computation  for  real-time  feasibility.  Therefore,  our  first 
step  in  implementing  quadric  surfaces  has  been  to  introduce  a  data  base 
constraint  to  maintain  computational  efficiency.  We  compose  our  data  base  as 
a  set  of  objects,  each  of  which  is  defined  by  one  quadric  surface  and  a  finite 
number  of  planes.  We  found  it  efficient  to  limit  the  number  of  planes  to  six. 
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although  there  is  no  theoretical  limit.  An  object  may  include  a  quadric  with 
no  bounding  planes  or  may  be  defined  by  planes  only.  Many  scene  features  can 
be  defined  by  a  single  object;  complicated  structures,  such  as  aircraft,  are 
defined  by  multiple  objects. 

The  simplicity  of  this  data  base  allows  for  efficient  techniques  to 
project  all  the  boundary  curves  onto  the  image  plane,  since  all  boundary 
curves  are  second-order.  The  quadric  silhouette  is  projected  as  the  limb 
curve,  and  each  curve  of  intersection  between  the  quadric  and  a  bounding  plane 
is  projected  as  an  intersection  curve.  The  nature  of  perspective  geometry 
also  allows  a  simple  computation  of  the  silhouette  plane  of  the  quadric 
surface.  We  have  developed  sophisticated  visibility  logic  using  the 
projection  of  the  intersection  between  a  bounding  plane  and  the  silhouette 
plane  to  define  a  visibility  line  on  the  image  plane.  The  visibility  lines 
are  used  to  delete  all  portions  of  the  limb  and  intersection  curves  which  are 
invisible.  Thus,  we  have  greatly  simplified  the  intraobject  visibility 
determination,  the  removal  of  hidden  surfaces  on  a  single  object,  by  using 
only  the  boundary  information. 

Interobject  visibility,  removal  of  hidden  surfaces  on  one  object  occulted 
by  another  object  is  even  simpler.  A  priority  list  of  objects  is  established 
for  each  frame,  initially  containing  only  those  objects  defined  in  the  data 
base  to  be  adjacent  (such  as  two  mountain  peaks  or  an  aircraft  wing  and 
fuselage).  The  bounding  plane  between  two  adjacent  objects  is  used  to 
determine  which  object  has  priority  relative  to  the  current  eyepoint.  On  a 
scan  line  basis,  interobject  visibility  is  tested  if  the  boundary  points  of 
two  objects  overlap.  The  test  first  checks  the  priority  list  to  resolve  the 
visibility.  If  the  two  objects  are  not  in  the  list,  test  of  distance  to  the 
silhouette  planes  is  used  and  the  priority  determination  is  added  to  the 
list. 

In  this  manner,  we  have  developed  a  unique  and  efficient  solution  to  the 
hidden  surface  problem  of  curved  surfaces.  Because  this  technique  operates  on 
a  scan  line  basis,  using  only  boundary  information,  it  reduces  the  problem  to 
a  simple  level  that  can  be  handled  in  real-time. 

Shading  of  Bounded  Quadric  Surfaces 

Our  technique  for  determining  the  visibility  of  quadric  surfaces  achieves 
its  efficiency  by  using  only  the  boundary  points  in  the  image.  Surface 
shading  must  be  computed  for  all  surface  points  in  the  image.  Since  the 
number  of  these  points  is  considerably  greater  than  the  number  of  boundary 
points,  a  brute  force  implementation  would  be  costly.  By  taking  advantage  of 
surface  coherence,  we  can  significantly  reduce  the  computation  of  smooth 
surface  shading.  We  have  developed  a  technique  that  uses  the  fact  that  the 
shading  due  to  diffuse  reflection  from  quadric  surfaces  varies  smoothly  along 
a  scan  line.  To  shade  a  quadric  surface  along  each  scan  line  we  compute  the 
shading  at  each  boundary  point  and  at  a  point  midway  between.  Interpolation 
between  these  points  produces  an  efficient  and  acceptable  approximation  to  the 
exact  shading.  Figure  1  shows  the  effectiveness  of  this  technique.  The  model 
of  the  KC-135  was  constructed  of  21  objects  requiring  a  total  of  532  data 
values  to  define  the  data  base.  To  produce  equivalent  quality  of  smooth 
shading  and  feature  definition,  edge  systems  would  require  a  data  base  an 
order  of  magnitude  greater  than  ours. 
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Texturing  of  Bounded  Quadric  Surfaces 

Although  quadric  surfaces  such  as  hyperboloids  provide  an  effective  basis 
for  modeling  major  terrain  features,  texturing  must  be  overlaid  to  represent 
natural  detail  necessary  to  produce  essential  flying  cues.  To  minimize  the 
scene  data  base  and  still  provide  a  variety  of  texture  patterns  representative 
of  a  mission  gaming  area,  we  have  developed  a  texturing  technique  that  uses  a 
mathematical  function  to  represent  the  statistical  characteristics  of  surface 
detail  with  a  limited  number  of  parameters  (4).  The  value  of  the  function, 
computed  at  each  pixel,  is  used  to  modulate  the  surface  shading.  Although 
this  is  the  most  computationally  intensive  technique  in  our  system,  we  have 
determined  that  real-time  implementation  is  feasible  at  an  acceptable  cost. 

An  important  feature  of  any  effective  texturing  technique  is  perspective 
validity,  that  is,  the  texture  pattern  must  be  fixed  to  the  surface  in  the 
scene  as  it  is  viewed  continuously  from  different  eyepoints.  We  achieve 
perspective  validity  by  defining  our  texturing  function  as  a  function  of  scene 
coordinates.  To  avoid  computing  the  scene  coordinates  for  each  pixel  we  use 
an  interpolation  technique  similar  to  that  used  for  shading.  Dynamic  image 
testing  has  confirmed  the  perspective  validity  provided  by  this  technique  and 
has  demonstrated  the  superior  flying  cues  produced  by  the  scene  detail 
generated  by  the  texturing.  Figure  2  shows  how  effectively  terrain  can  be 
modeled  by  quadric  surfaces  with  scene  content  enhanced  by  texturing.  The 
scene  consists  of  four  hyperboloids  and  a  ground  plane  with  texturing 
generated  by  a  23-parameter  function. 

The  power  of  our  functional  approach  to  texturing  is  demonstrated  further 
in  Figures  3  and  4.  Because  we  can  control  the  function  completely  as  we 
generate  the  texturing,  we  can  add  variable  translucence  to  produce  natural 
features  with  very  irregular  boundaries,  such  as  clouds  and  trees.  Control  of 
the  function  can  be  implemented  dynamically  to  produce  frame-to-frame 
variations  to  simulate  wind  effects  on  the  clouds  or  trees. 

PROGRESS  ON  AVSS 

To  develop  a  next-generation  CIG  system  free  of  the  limitations  of 
current  edge  systems,  the  Air  Force  has  undertaken  the  three  phase  AVSS 
project.  Phase  I  includes  development  of  advanced  CIG  techniques  and  an 
overall  system  concept.  Phase  II  will  include  software  development,  image 
testing,  and  hardware  design.  Phase  III  will  involve  construction  of  a 
real-time  prototype  system.  Grumman's  effort  on  Phase  I  has  included  the 
investigation  of  candidate  techniques  to  provide  the  features  listed  in  Table 
1.  Based  on  our  investigation  we  chose  the  best  technique  for  each  feature 
and  developed  a  system  concept  to  Incorporate  all  chosen  techniques  In  a 
feasible  real-time  system.  In  the  implementation  of  each  feature,  the 
technique  we  developed  represents  a  significant  advancement  In  CIG  technology. 
We  will  describe  the  system  concept  and  some  of  the  most  significant 
techniques. 
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System  Concept 

The  standard  system  approach  in  current  CIG  systems  is  a  large  pipeline 
which  implements  all  of  the  equations,  tests,  and  flow  of  a  particular  set  of 
techniques.  This  results  in  a  massive  special-purpose  processor  with  many 
different  types  of  modules.  Typically,  circuit  boards  must  be  wire-wrapped, 
and  interconnection  of  boards  is  a  problem.  A  chan;  in  the  features  requires 
changes  or  additions  to  the  hardware.  Inis  type  of  large  system  has  frequent 
failures  and  is  extremely  difficult  to  r'.rdr  and  maintain.  Current  systems 
use  TTL  and  even  ECL  which  dissip-v-  .>•:•  amounts  of  power. 

Figure  5  shows  a  block  diagram  of  cur  system  architecture.  The  basic 
premise  of  this  design  is  to  partition  the  processing  to  be  performed  into 
tasks  which  can  make  effective  use  of  parallelism.  These  tasks  are  then 
pipelined  at  the  frame  level.  Tasks  may  be  divided  into  subtasks  which  are 
performed  by  an  array  of  processing  elements  operating  in  parallel.  Tasks  are 
chosen  to  minimize  interprocessor  communication. 

There  are  two  critical  tasks  which  must  be  partitioned.  The  first  is  to 
transform,  project  and  remove  hidden  curve  segments  for  each  object  in  the 
scene.  This  task  is  performed  by  a  number  of  processors  which  we  call  object 
processors.  This  task  partitions  very  efficiently  -  each  object  processor 
computes  one  object  in  the  scene.  Interprocessor  communication  within  this 
task  is  not  needed  because  all  operations  can  be  performed  on  a  given  object 
with  no  information  about  other  objects  in  the  scene. 

Once  the  curve  and  surface  information  is  determined  for  each  object,  the 
entire  set  of  objects  must  be  rendered  with  interobject  visibility  determined. 
This  task  is  partitioned  by  dividing  the  frame  into  subframes,  with  each 
subframe  computed  by  a  separate  processor.  This  is  an  appropriate 
partitioning  because,  while  there  are  many  objects  in  a  scene,  the  number  of 
objects  overlapping  a  subframe  can  be  limited  to  a  small  number.  With  only  a 
little  overhead  for  scan  line  intersection  and  antialiasing  at  the  subframe 
border,  interprocessor  communication  at  this  level  is  unnecessary. 

Each  object  processor  determines  which  subframes  require  its  object 
information  and  routes  object  information  over  a  bus  connecting  object  and 
subframe  processors. 

Object  information  enters  the  object  processors  from  a  standard  mainframe 
computer.  This  computer  takes  information  from  a  simulator  console  or  flight 
simulator  computer  and  determines  which  objects  are  in  the  current  scene.  The 
scene  is  paged  into  memory  from  disk  as  a  function  of  eyepoint  and  look  angle. 
Object  processors  make  use  of  frame  coherence  in  that,  if  an  object  is  present 
in  two  successive  frames,  it  is  processed  by  the  same  object  processor.  Thus, 
the  mainframe  computer  needs  only  to  transmit  new  objects  to  object  processors 
as  they  appear  and  keep  track  of  object  processors  which  can  be  reclaimed. 

This  limits  the  bandwidth  between  the  mainframe  and  the  object  processors. 

At  the  output  of  the  subframe  processors,  the  separate  subframes  are 
pieced  together  and  converted  Into  a  raster  format  for  an  RGB  display.  The 
single  output  signal  is  the  highest  bandwidth  channel  in  the  system. 
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There  are  many  advantages  of  the  proposed  system  architecture.  Because 
of  efficient  task  partitioning,  the  problem  has  been  defined  in  terms  of  many 
identical  modules,  each  of  which  operates  at  a  fraction  of  the  total  system 
throughput.  Since  the  modules  are  identical,  they  can  be  implemented  in  a 
cost-effective  manner  using  LSI  technology.  Each  processor  will  be  smaller, 
cheaper,  and  use  less  power  when  Implemented  in  this  way.  Cost  can  be 

minimized  by  trading  off  the  throughput  of  each  processor  as  a  function  of 
technology,  and  the  size,  number  and  complexity  of  each  processor.  The  system 
is  flexible  since  system  parameters  such  as  number  of  objects  or  size  of  frame 
can  be  varied  by  changing  the  number  of  modules.  Additionally,  each  processor 
is  programmable,  so  that  system  features  can  be  altered,  extended  or 
customized  simply  by  updating  the  software.  Such  a  system  is  much  more 
resistant  to  obsolescence  than  hardware-oriented  pipeline  systems. 

For  prototype  debugging,  our  risk  is  minimized  since  we  can  build  and 
debug  a  single  object  and  subframe  processor.  These  processors  can  be  driven 
by  a  general  purpose  computer  to  produce  a  full  image  for  testing  before  the 
final  design  is  committed.  At  that  point,  the  processors  can  be  replicated  to 
build  a  real-time  system. 

Surface  Visibility 

Me  refined  our  intraobject  visibility  technique  greatly  by  taking 
advantage  of  scan-line  coherence.  Our  refined  technique  will  be  implemented 
in  each  object  processor  and  will  use  the  computed  visibility  lines  of  an 
object  to  determine  all  visible  boundary  portions  and  those  scan  lines  on 
which  each  portion  is  visible.  The  object  processor  will  pass  this  visibility 
information  to  the  appropriate  subframe  processors  which  will  then  compute 
only  the  visible  boundary  points.  This  represents  a  significant  increase  in 
processing  efficiency  over  our  old  technique,  which  required  computation  of 
all  boundary  points  on  a  scan  line,  testing  the  visibility  of  each,  and 
deleting  those  that  were  hidden.  The  effectiveness  of  our  new  technique  is 
demonstrated  by  Figures  6  through  9.  These  figures  also  demonstrate  the 
complexity  of  the  objects  that  can  be  modeled  using  quadric  and  planar 
surfaces. 

Tonal  Computation 

We  refined  our  technique  for  diffuse  shading  by  extending  the  scan  line 
interpolation  to  interpolation  across  scan  lines  as  well.  An  interpolation 
function  will  be  computed  for  a  surface  in  the  object  processor  and  passed  to 
the  appropriate  subframe  processors.  The  subframe  processor  computation  will 
be  limited  to  incremental  calculation  of  the  function  along  each  scan  line 
with  incremental  updating  of  interpolation  function  coefficients  from  scan 
line  to  scan  line.  A  significant  benefit  of  this  refinement  is  that  it 
eliminates  the  need  for  projecting  into  scene  coordinates  during  scan  line 
processing. 

Specular  reflection,  which  provides  valuable  cues  for  target  detection. 

Is  much  more  complicated  to  model  than  diffuse  reflection  and  is  consequently 
not  included  as  a  feature  of  current  systems.  We  extended  our  diffuse  shading 


interpolation  technique  to  provide  an  efficient  approximation  to  the  physics 
of  specular  reflection.  A  good  mathematical  model  for  specular  reflection 
involves  an  expression  of  three  vector  dot  products,  one  of  which  is  the  dot 
product  used  for  diffuse  reflection.  Our  analysis  has  shown  that  the  other 
two  dot  products  vary  across  a  surface  almost  as  smoothly  as  the  diffuse 
shading  dot  product.  We  will  therefore  approximate  each  dot  product  by 
interpolation  and  compute  the  expression.  The  model  for  specular  reflection 
requires  raising  the  expression  to  a  power  as  high  as  10,  but  this  can  be  done 
efficiently  by  using  an  exponent  that  is  a  power  of  2  or  by  using  a  look-up 
table.  The  added  computation  for  this  technique  is  acceptable  and  should 
provide  an  effective  representation  of  the  essence  of  glint  from  windows, 
still  water  and  metallic  surfaces. 

Antialiasing 

The  ideal  solution  for  antialiasing  an  image  is  to  filter  out  all  spatial 
frequencies  higher  than  half  the  spatial  sampling  frequency.  We  developed  a 
prefiltering  technique  that  approximates  the  ideal  solution  very  effectively 
by  applying  a  non-uniform  weighting  to  the  image  over  a  region  covering  two 
sampling  periods  in  the  horizontal  and  vertical  directions  (5).  Current  CIG 
systems  use  an  area-weighted  average  for  each  pixel.  This  technique  applies  a 
uniform  weighting  over  one  sampling  period  and  is  therefore  a  poor 
approximation  to  the  ideal  solution.  Figure  10  shows  a  test  image  of  a  set  of 
annular  surfaces  of  one-half-pixel  width  with  severe  aliasing.  Figure  11 
shows  the  result  of  antialiasing  using  the  current  area-weighted  average 
technique.  Figure  12  shows  considerable  improvement  in  antialiasing  using  our 
technique.  Although  our  technique  greatly  outperforms  that  of  current 
systems,  it  is  much  less  costly  to  compute. 

Scene  Control 


One  of  the  significant  problems  for  any  CIG  system  is  level  of  detail 
(LOD)  management  to  avoid  system  overload.  Our  compact  data  base,  using 
textured  quadric  surfaces,  minimizes  this  problem  by  defining  scene  features 
as  efficiently  as  possible.  In  addition,  the  nature  of  our  object  modeling 
allows  us  to  implement  LOD  management  effectively  with  only  two  levels  of 
detail.  The  high  level  of  detail  will  consist  of  features  represented  in  full 
detail.  The  low  level  of  detail  will  consist  of  a  single  object  representing 
a  cluster  of  these  features.  For  example,  a  forest  will  be  represented  in 
high  LOD  as  a  large  number  of  individual  trees  and  in  low  LOD  as  a  single 
textured,  bounded  quadric.  At  long  distances,  only  the  low  LOD  forest  will  be 
presented.  At  short  distances,  the  high  LOD  trees  will  be  shown.  To 
transition  smoothly  as  the  viewpoint  approaches  a  distant  forest  we  will 
introduce  a  bounding  plane  to  truncate  the  forest  on  the  near  end  (Figure  13). 
The  plane  will  be  constructed  at  a  fixed  distance  from  the  viewpoint  and  will 
continuously  truncate  the  low  LOD  object  as  the  eye  approaches  until  the  LOD 
object  is  completely  eliminated.  Meanwhile,  the  high  LOD  objects  just  beyond 
the  plane  (inside  the  low  LOD  object)  will  appear  with  diminishing 
translucence.  As  the  high  LOD  objects  emerge  In  front  of  the  bounding  plane, 
their  translucence  will  be  decreased  till  they  are  completely  visible.  The 
process  would  be  reversed  for  viewpoint  motion  away  from  the  forest.  This 
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technique  will  introduce  minimal  data  base  overhead  while  providing  effective 
scene  control  with  smooth  transitions  to  prevent  overload. 

DMA  Elevation  Modeling 

The  greatest  problem  involved  in  modeling  with  quadric  surfaces  is  to 
model  arbitrary  topographical  data,  such  as  the  Defense  Mapping  Agency  (DMA) 
elevation  data,  with  adequate  surface  continuity.  However,  modeling  the  DMA 
data  base  with  edges  has  produced  unacceptable  representations  characterized 
more  by  linear  artifacts  than  by  natural  contouring.  Our  approach  to  this 
problem  has  been  to  model  the  significant  terrain  features  with  quadric 
surfaces  and  to  represent  the  secondary  terrain  fluctuations  with  texturing. 

We  have  divided  our  approach  into  the  following  subtasks: 

o  defining  the  significant  features 
o  isolating  the  DMA  data  samples  for  each  feature 
o  fitting  a  quadric  surface  to  the  data  samples  for  each  feature 
o  adding  bounding  planes  to  each  quadric  to  maximize  surface 
continuity  between  features 

o  determining  texturing  function  parameters  to  represent 
secondary  topographical  variations  in  the  DMA  data. 

In  a  limited  empirical  study  using  actual  digitized  elevation  data,  we 
developed  a  technique  using  low-pass  filtering  to  reduce  the  data  to  its 
significant  features,  which  can  be  predefined  by  the  choice  of  filter 
bandwidth. 

To  isolate  the  significant  features  in  the  filtered  data  we  developed  a 
technique  which  applies  the  Laplacian  operator  to  the  data  to  detect  points  of 
extreme  curvature  at  the  feature  boundaries.  We  also  developed  a  technique 
analyzing  the  data  profile  by  profile  and  using  profile  coherence  within 
features. 

Once  the  feature  sample  points  are  located,  fitting  a  quadric  surface  by 
least  squares  minimization  is  straightforward.  Computation  time  is  acceptable 
in  view  of  the  fact  that  the  modeling  will  be  done  off-line. 

The  equations  of  the  quadrics  fitted  to  the  features  provide  the 
information  necessary  to  fit  bounding  planes  at  the  juncture  of  adjacent 
features.  Our  technique  allows  construction  of  layers  of  features,  one  on  top 
of  the  other,  to  model  complicated  terrain. 

The  texture  function  parameters  will  be  based  on  the  frequency  content  of 
the  original  data  above  the  low  pass  filter  cutoff,  i.e.,  those  frequencies 
not  included  in  the  quadric  surface  determination. 

In  this  manner,  we  will  model  the  major  topographical  features  of  the  DMA 
elevation  data  explicitly,  and  we  will  represent  the  minor  topographical 
variations  statistically.  This  approach  has  significant  advantages  over 
current  edge  techniques.  First  it  will  produce  a  far  more  compact  data  base. 
Second,  it  models  the  data  as  a  set  of  features  as  the  eye  would  perceive  the 
scene  and  therefore  provides  an  optimum  representation  for  visual  cues. 
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Third,  the  filtering  and  surface  fitting  operations  supress  random  errors 
which  are  known  to  be  present  in  the  DMA  data.  Finally,  as  shown  in  Figure  2, 
textured  quadric  surfaces  produce  a  far  more  faithful  representation  of 
natural  nonlinearity  and  detail  than  is  possible  using  edges. 

Other  AVSS  Features 


Other  system  features  are  significantly  enhanced  by  our  ability  to  model 
with  quadric  surfaces.  Scene  modeling  is  simpler  and  more  effective  due  to 
our  ability  to  represent  curvature  so  efficiently.  This  is  true  of  cultural 
features  as  well  as  natural  features,  as  shown  in  Figures  1  through  4. 
Implementation  of  moving  models  is  simplified  because  there  is  less  data  to 
which  transformation  of  coordinates  must  applied.  Greater  capability  for 
special  effects  is  provided  by  textured  quadrics,  which  can  efficiently  model 
clouds,  puffs  of  smoke,  and  balls  of  fire.  Our  capability  for  dynamic 
texturing  will  enhance  the  realism  of  these  effects  to  provide  cues  of  wind 
and  weapons  effects.  In  addition,  our  ability  to  handle  curved  boundaries  in 
the  image  will  facilitate  representing  shadows  as  simply  bounded  regions  on  a 
surface  and  lights  of  finite  size.  Our  antialiasing  technique  will  produce  an 
enhanced  representation  of  point  light  sources  free  from  distracting 
scintillation  anomalies. 

CONCLUSION 

Textured  quadric  surfaces  provide  an  efficient  and  effective  means  of 
scene  modeling.  They  present  greater  scene  content  of  higher  quality  than 
that  produced  by  edge  systems.  The  many  fundamental  advantages  in  this  form 
of  modeling  allow  cost-effective  solutions  to  many  of  the  problems  that 
degrade  current  systems.  At  the  same  time,  quadrics  are  far  simpler  and 
cheaper  to  implement  than  higher-order  surfaces. 

Our  goal  has  been  to  find  and  exploit  the  most  efficient  means  of 
providing  essential  cues  to  provide  more  valid  training.  We  believe  that 
quadric  surfaces  with  texturing  are  the  most  efficient  means,  for  they 
epitomize  a  fundamental  truth  stated  by  Einstein:  "Everything  should  be  as 
simple  as  possible,  but  no  simpler." 
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ABSTRACT 


The  recent  needs  in  aircraft  training,  spatial  flights 
have  increased  the  studies  of  flight  simulators,  with  realistic  real-time 
picture  animation. 

These  pictures  may  be  : 

-  aircrafts,  space-shuttle,  vehicles,  landscapes,  airports,... 

One  of  the  missions  of  the  CENTRE  D* ELECTRON I QUE  DE  L'ARMEMENT 
is  military  systems  assessment. 

In  order  to  evaluate  military  aircrafts  it  was  build  a  dos 
flight  simulator.  One  of  the  enhancement  decided  in  1977  for  this  simulator 
consists  in  a  video  synthetic  image  generator  for  thf  target  projection. 


KEY-WORDS 


-3D  graphics, 

-  raster-scan  displays, 

-  computer  aided  design  3  D, 

-  simulation. 
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1  -  INTRODUCTION 

The  objectives  of  the  generator  were  : 

-  definition  :  512  X  512, 

-  brightness  levels  :  256, 

-  regeneration  time  for  each  image  :  40  mS, 

-  output  :  Red  Green  Blue  coded  vis  a  look-up  table  of 
256  words  of  24  bits, 

-  complexity  of  the  object  :  up  to  400  polygons, 

-  objects  :  aircrafts,  helicopters,  vehicles,... 

The  goal  of  this  paper  is  to  describe  : 

a/  The  concepts  of  this  CIG  system,  primarily  used  for  re¬ 
presentation  of  realistic  objects  in  aircraft  simulators 

b/  The  design  and  architecture  of  this  system  and  its 
performances. 

c/  The  different  applications  of  this  system. 


2.1.  -  THE  CONCEPTS  OF  THE  CIG  SYSTEM 

In  order  to  realize  a  system  giving  these  possibilities  in 
real  time,  it  was  necessary  to  find  a  performing  algorithm  of  hidden 
faces  elimination,  able  to  be  implemented  in  hardware. 

In  1976,  at  the  biginning  of  these  studies  the  basic 
algorithms  were  those  of  : 

1/  WATKINS 

2/  WARNOCK 

3/  SHUMACHER. 

A  simulation  of  algorithm  1  and  the  study  of  the  second 
showed  that  it  would  be  very  difficult  to  wire  them. 

The  third  algorithm,  however,  brought  the  following  idea  : 

It  was  possible  to  prepare  before  the  real-time  display  a 
list  of  the  polygons  of  the  object,  called  PRIORITY  LIST.  This  list 
is  obtained  by  a  sort  among  the  polygons  from  the  farest  to  the  nearest 

When  the  object  is  displayed  in  real-time  it  suffices  to 
send  the  polygons  on  the  screen  in  the  same  order,  just  as  a  painter 
which  would  paint  first  would  paint  first  the  farest  plans,  and  would 
superpose  the  nearer  ones. 

The  difficulty  lies  in  the  case  of  complex  objects.  In  this 
case,  the  priority  list  is  not  unique. 

When  a  face  1  hides  face  2,  face  2  can  hide  face  1  when  we 
turn  around  the  object. 

Scientifically  speaking,  the  graph  of  the  priorities 
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presents  cycles,  and  we  have  no  list  of  priority  independently  of  the 
point  of  view. 

In  order  to  cut  these  cycles,  we  have  2  solutions  : 

-  separate  the  object  in  clusters  by  planes,  each  cluster 
having  a  priority  list  without  cycle, 

-  separate  the  space  in  viewport  solid  angles,  so  that 
in  each  solid  angle  no  cycle  can  appear,  when  we 
change  the  point  of  view. 

This  method  can  be  implemented  : 

-  first  by  defining  a  list  of  solid  angles, 

-  then  by  computing  priority  lists  in  each  one. 

The  corresponding  program  operates  in  the  following 

manner  : 


-  the  determination  of  solid  angles  is  obtained  by 
testing  one  by  one  all  the  polygons  of  the  object  : 
the  program  retains  only  those  wich  minimize  cycles 
in  the  priority  list, 

-  after  that,  from  a  given  list  of  solid  angles  the  program 
computes  priority  list  in  each  solid  angle. 

If  there  are  another  cycles  after  this  phase,  the 
program  introduces  new  solid  angles,  until  no  cycle 
appears. 

In  order  to  test  the  priority  of  a  facet  a  with  a  facet  B, 
we  have  determined  a  criterion  by  means  of  scalar  products  be* ween 
the  normal  of  facet  A  and  the  segments  joining  one  vertex  to  the 
vertices  of  B. 

The  result  of  this  program  is  a  set  of  N  priority  lists 
corresponding  to  the  N  solid  angles. 

2.2.  -  SHADING  AND  SMOOTHING 

The  polygons  of  the  objects  must  be  : 

-  Planar, 

-  Convex. 

Their  color  may  be  fixed  (simple  shadowing)  or  variable 
in  light  intensity  (smooting) . 

So,  it  is  possibl<  to  display  real  edges  with  uniform 
shadowing  and  curved  surfaces  with  smoothing.  An  interpretation  of 
the  algorithm  of  H.  GOURAUD  has  been  wired  in  the  hardware  of  the 
system. 
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2.3.  -  ROTATE  THE  SCENE 

The  rotation  of  the  scene  is  realized  by  a  bit-slice 
microprocessor  and  a  fast  multiplier  TRW. 

REMARK1:  In  this  first  version,  we  have  implemented 
neither  conic  projection,  nor  clippins  :  it  was,  in  fact  no  useful 
to  generate  aircrafts,  vehicles  or  simplified  scenes. 


3  ~  DESIGN  AND  ARCHITECTURE  OF  THE  SYSTEM 

After  the  object  has  been  digitized,  we  compute  solid 
angles  and  priority  lists.  So  it  remains  in  real  time  to  : 

3.1.  -  CALCULATE  SHADING 

3.2.  -  SMOOTH  FACETS 

3.3.  -  ROTATE  AND  TRANSLATE  THE  SCENE 

3.4.  -  DETERMINE  THE  GOOD  PRIORITY  LIST  CORRESPONDING  TO 

THE  POSITION  OF  THE  OBSERVER 

3.5.  -  SUPERPOSE  FACETS  IN  AN  IMAGE  MEMORY,  FOLLOWING  THE 

INCREASING  ORDER  OF  PRIORITY  (GIVEN  BY  THE  PRIORITY 
LIST) 


These  tasks  are  realized  by  2  hardware  modules  : 

a/  MICROPROCESSOR  MODULE  : 

Built  with  AMD  2900  series 
with  : 


4  K  of  64  bits  words  of  microprogram  memory, 
16  K  to  32  bits  words  of  DATA  memory. 

The  machine  built  contains  only  1  microprocessor  but  the 
architecture  and  the  bus  for  exchanges  allows  an  extension  to  2 
microprocessors  with  their  own  memory. 

The  micro-instruction  cycle  time  is  about  200  nS, 

The  microprogram  constitutes  the  program  of  the  module 
(there  is  no  program  memory) . 

During  an  instruction,  the  microprocessor  can  execute  : 

-  a  logical  or  arithmetical  test, 

-  or  an  arithmetic  operation, 

-  or  a  branch  (conditionnal  or  not) . 

Or  the  three  operations  in  the  same  time. 
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The  input  of  this  module  is  : 


-  The  data  base  of  the  scene ,  loaded  in  the  DATA  memory 
by  a  main  frame-computer  before  real-time  display, 

-  coordinates  of  the  vertices  of  the  scene, 

-  description  of  the  facets  (or  polygons)  of  the  scene, 

-  list  of  normals  of  the  polygons, 

-  a  set  of  priority  list. 

-  The  dynamic  parameters  of  the  scene  : 

-  rotation  matrix  of  the  scene, 

-  the  scale  factor  of  the  scene, 

-  the  direction  of  the  light  source. 

These  parameters  are  sent  in  real-time  by  a  main-frame 
computer  which  knows  the  relative  position  of  the  objects  in  the 
space  (for  example  :  aircrafts,  helicopters,...) 

In  real  time  the  microprocessor  computes  first  : 

3.1.-  SHADOWING 

For  the  shadowing  we  use  the  following  formula  : 

S  =  255  X  (COS  (TETA)  +  1) 

Where  TETA  is  the  angle  between  light  source  and  the 
normal  of  the  facet. 


3.2.  -  SMOOTHING  FACETS 


We  compute  at  each  vertice  of  the  object  : 

(SI  +  S2  +  S3  +  .  S  n)/N 

Where  SI  ...  Sn  represents  the  shadows  of  the  facets 
containing  this  vertice. 


3.3.  -  ROTATE  AND  TRANSLATE  THE  SCENE 


The  microprossor  computes  : 

V'  =  M  X  V  +  T  where  : 

M  is  the  rotation  matrix  of  the  object 
v  represents  the  coordinates  of  a  vertex 
T  represents  the  translation  vector. 
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3.4.  -  RESEARCH  OF  THE  GOOD  PRIORITY  LIST 

Corresponding  to  the  solid  angle  containing  the  observer. 


3.S.  -  FACETS  LOADER 

After  tasks  1  to  4,  the  microprocessor  sends  to  a 
FACET  GENERATOR  MODULE  the  description  of  each  facet  from  the 
farest  to  the  nearest  according  to  the  order  of  the  priority  list. 

Each  facet  is  decomposed  in  trapezoidal  blocks  which 
are  defined  by  : 


-  position  XG,  XD  of  the  beginning  of  edges  right  and 
left , 

-  number  of  lines  of  the  block, 

-  slopes  of  the  edges  right  and  left, 

-  shadows  of  the  extremities  left  and  right  of  the 
first  segment, 

-  slopes  of  the  shadows  left  and  right, 

-  position  Y  of  the  first  segment. 

These  datas  are  sent  to  the  FACET  GENERATOR  MODULE 
which  superposes  the  facets  in  an  image  memory. 


b/  FACET  GENERATOR  MODULE  : 

This  module  is  entirely  in  hardware. 

The  tasks  accomplished  by  this  module  are  : 

-  From  a  line  to  the  following  : 

-  increment  XLEFT  and  XRITHT  : 

XLEFT  =  XLEFT  +  DXLEFT 
XRIGHT  =  XRIGHT  +  DXRIGHT 

-  increment  S LEFT  and  SRI 3HT  : 

SLEFT  =  SLEPT  +  DSLEFT 
SRIGHT  =  SR1GHT  +  DSRIGHT 

-  decrement  DY  : 

DY  =  D  -  1 

-  Inside  of  a  line  : 

-  computation  of  the  slope  of  shadow  : 

DS  =  (SRIGHT  -  SLEFT)  /  (XRIGHT  -  XLEFT) 

-  computation  of  the  current  shadow  : 

S  =  S  +  DS. 
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The  cycle  time  of  the  module  is  :  100  nS/pixel. 

The  facet  generator  module  outputs  pixels  through  an 
image  memory  whose  size  is  : 

512  X  512  X  8  bits. 

There  are  2  image  memories  which  are  filled  in  flip-flop. 

The  information  (8  bits  per  pixel)  is  sent  to  the  TV 
projector  through  D/A  converters. 

At  the  same  time,  the  same  information  is  sent  to  a 
pseudo-color  table  of  256  words,  and  via  3  D/A  converters  to 
a  Red  Green  Blue  monitor. 


4  -  PERFORMANCES  OF  THE  CIG  SYSTEM 

-  The  CIG  system  can  display  in  real  time  up  to  400 
facets  of  medium  size  with  the  microprossor  version. 

-  The  medium  time  necessary  to  compute  one  pixel  is 

about  100  nS. 


-  The  performances  in  bi-processor  version  have  not 
yet  been  tested.  However  we  can  hope  a  gain  of  about  100  %  in  facets. 

-  If  POLYNUM  is  the  number  of  polygons,  the  computation 
time  of  the  object  is  given  by  : 

T  =  0.200  X  897  X  POLYNUM  if  the  polygons  are  smoothed 

T  =  0.200  X  585  X  POLYNUM  if  the  polygons  are  not  smoothed 

(This  time  is  given  for  an  instruction  cycle  time  of 

200  nS) . 


5  ~  THE  DIFERENT  APPLICATIONS  OF  THE  CIG  SYSTEM 

The  first  generation  of  CIG  system  (GSI  1)  was  operationnal 

in  1980  may. 


The  applications  of  this  system  are  numerous  : 

-  generation  of  aircrafts,  tanks,  helicopters,  images..., 

-  recognition  of  the  attitude  of  an  aircraft  by  comparison 
between  a  photography  and  a  synthetic  image, 

-  out  of  the  simulation  domain,  we  can  mention  : 

-  representation  of  forms  in  design  engeneering  : 
(architecture,  urbanism,  vehicles,  aircraft  design), 

-  medical  an  chemical  application, 

-  movies 

-  TV  and  Audio  Visual  production, 

-  microinformatics. 
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Low-Cost  Air-to-Ground  Weapons  Delivery  Simulation 
Using  Microprocessor-Driven  Displays 

ABSTRACT 

A  low-cost  flight  simulation  facility  for  helicopter  gunship  air-to- 
ground  weapons  system  simulation  has  been  developed  under  U.  S.  Army  Missile 
Command  contract  DAAK40-79-C-0060  by  The  University  of  Alabama.  The  system 
encompasses  interfacing  the  following  features  into  the  Electro-Optical 
Simulation  System  (EOSS)  to  constitute  an  Air-to-Ground  Weapons  Delivery 
Simulator: 

1.  Distributed  microprocessor  (host  PDP11/34)  color  graphics  system 
for  video  instrument  panel  and  cockpit  window  graphics. 

2.  Surplus  Apollo  program  optics  adapted  for  high-resolution  cockpit 
window  displays. 

3.  Weapons  special  effects  and  trajectory  software  package. 

4.  Fully  instrumented  separate  pilot  and  gunner  cockpits. 

5.  General-purpose  fire-control  system  console. 

I.  INTRODUCTION 

In  the  spring  of  1979,  technical  personnel  of  the  Army  Missile  Command's 
EOSS  (Electro-Optical  Simulation  System)  facility  were  presented  the  problem 
of  visually  simulating  a  tracer  trajectory  for  a  high-velocity  projectile. 

The  solution  of  this  problem  led  to  the  development  of  a  Low-Cost  Air-to- 
Ground  Weapons  Delivery  Simulation  (WDS)  facility.  The  major  portion  of  the 
work  was  performed  under  contract  DAAK40-79-C-0060  by  The  University  of 
Alabama.  Due  to  austere  funding,  some  of  the  major  subsystems  were  fabri¬ 
cated  from  excess  government  property  collected  over  the  past  several  years, 
which  kept  the  cost  for  the  WDS  development  under  two  hundred  thousand 
dollars. 

The  need  for  the  development  of  the  WDS  was  brought  about  by  an  ex¬ 
ploratory  development  program  for  a  high-velocity  kinetic  energy  projectile 
to  be  launched  from  a  helicopter  gunship.  The  scenario  involves  a  pilot 
bringing  his  aircraft  up  from  cover,  locating  an  enemy  target  (generally  an 
armored  vehicle),  maneuvering  his  gunship  in  line  with  the  target,  firing  a 
burst  or  volley  of  projectiles,  observing  points  of  impact  and  miss- 
distances,  realigning  his  gunship  and  firing  a  second  volley  of  projectiles. 
(This  scenario  can  be  repeated  if  necessary  and  if  time  permits  before  re¬ 
turning  to  cover.)  Each  projectile  contains  a  tracer  material  so  that  the 
gunship  pilot  can  observe  the  trajectory  of  the  round  and  its  impact  point. 
The  accuracy  of  the  projectile's  trajectory  is  affected  not  only  by  the 
aiming  error  of  the  pilot  t.at  by  the  "noisy"  environment  of  the  helicopter 
serving  as  a  launch  platform.  This  hostile  launch  environment  led  to  the 
need  to  develop  a  capability  for  simulation  of  helicopter-pilot  dynamics 
along  with  their  resulting  impact  on  the  projectile  trajectories. 

The  solution  to  the  problem  was  to  develop  a  microprocessor  Video 
Display  System  (VDS)  which  would  generate  the  tracer  trajectories  and 
through  video  techniques  overlay  the  trajectories  on  television  displays 
of  the  target  scene  provided  by  the  EOSS  terrain  model.  A  functional  block 
diagram  of  the  Weapons  Delivery  Simulation  (WDS)  facility  is  shown  in 
Figure  1.  The  basic  subsystems  are: 
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Helicopter  Simulation  I  1  PDP  H/34 


FIGURE  1  -  WEAPONS  DELIVERY  SYSTEM  BLOCK  DIAGRAM 


L.  Helicopter  cockpit  mock-up  with  pilot  and  copilot-gunner  stations 
including  collimated  out-the-windscreen  display,  color  TV  instrument  panel 
and  full  flight  controls  for  helicopter  flight  simulation. 

2.  EOSS  6-DOF  motion  simulation  including  a  3-DOF  flight  table  with 
vertical  and  lateral  motion  capability  and  a  scaled  moving  terrain  model 
for  range  closure. 

3.  Helicopter  dynamics  simulation  generated  on  an  AD/4  analog 
computer. 

4.  PDP11/34  host  computer  for  generating  trajectory  data  and  flight 
instrument  panel  data. 

5.  Two  microprocessor  video  display  systems  (VDS) :  one  (VDS  #1)  for 
generating  tracer  trajectories  and  other  special  effects  for  overlaying  on 
the  windscreen  display,  and  the  other  (VDS  #2)  for  generating  the  cockpit 
instrument  panel  displays. 

6.  A  video  special  effects  generator  for  mixing  the  output  of  VDS  #1 
with  the  video  from  the  EOSS  TV  camera. 

The  EOSS  facility,  including  the  AD/4  analog  computer  and  a  PDP11/20 
minicomputer,  were  the  only  hardware  equipment  on  hand  at  the  outset  of  the 
program.  The  PDP11/34  minicomputer  and  video  special  effects  generator  were 
purchased  as  off-the-shelf  items.  The  two  cockpit  stations  including  the 
windscreen  infinity  displays,  flight  controls  and  instrument  panel  were 
fabricated  from  excess  hardware,  and  the  two  VDS  units  were  conceived,  de¬ 
signed  and  fabricated  by  the  contractor.  All  video  display  software  for 
programming  the  PDP11/34  and  the  two  VDS  units  had  to  be  developed  and 
checked  out.  The  helicopter  simulation  software  had  to  be  adapted  from  a 
more  complex  simulation  in  order  to  fit  the  AD/4  computer. 

The  following  sections  describe  the  general  concept  of  the  Weapons 
Delivery  Simulation  concept  and  the  subsystems  which  comprise  the  facility. 

II.  WEAPONS  DELIVERY  SYSTEM 

Cockpits 

The  cockpits  of  the  WDS  were  made  from  a  wooden  COBRA  mock-up 
salvaged  from  a  HELLFIRE  contract.  The  normal  tandem  configuration  was 
abandoned  in  favor  of  providing  two  side-by-side  units  with  each  having  its 
own  infinity  windscreen  displays,  instrument  panels  and  appropriate  con¬ 
trols.  In  Figure  2,  the  unit  on  the  left  is  the  pilot's  cockpit  and  con¬ 
tains  flight  controls  salvaged  from  a  wrecked  UH-10.  Potentiometers  mounted 
on  these  flight  controls  provide  helicopter  control  inputs  to  the  helicopter 
simulation  on  the  AD/4  analog  computer.  To  provide  aural  realism,  helicopter 
sound  generators  are  provided  in  each  cockpit. 

Partial  flight  instrument  data  is  displayed  to  both  the  pilot  and  the 
copilot/gunner  stations  on  separate  12-inch  color  graphics  television  dis¬ 
play  monitors  which  have  an  eight -megahertz  bandwidth  for  greater  horizontal 
resolution  capability.  Typical  flight  data  displayed  includes  velocity, 
altitude,  percent  throttle,  pitch-roll  attitude  (artificial  horizon)  and 
side  acceleration.  Other  data  can  be  readily  programmed  and  displayed 
through  the  host  minicomputer  and  VDS  microprocessor  #1. 

Each  cockpit  has  its  own  color  TV'/mirror/beamsplitter  infinity  wind¬ 
screen  display.  The  infinity  displays  were  considered  mandatory  for  the 
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EOSS  simulation  so  that  look-angle  independence  from  head  motion  could  be 
achieved  and  inside-outside-the-cockpit  pilot  visual  lags  could  be  ade¬ 
quately  simulated.  The  low  development  cost  requirement  dictated  that  a 
standard  525-line  color-video  monitor  be  used.  This  requirement  in  con¬ 
junction  with  the  high  resolution  requirement  of  air-to-ground  weapons  de¬ 
livery  simulation  limited  the  field  of  view  of  the  windscreen  displays  to 
25  degrees.  This  narrow  windscreen  f ield-of-view  does  not  limit  the  range 
of  research  expected  to  be  performed  in  the  EOSS.  Pilot  handling  qualities 
at  moderate-to-slow  forward  speeds  are  adequate  and  the  target  detection/ 
acquisition/engagement  task  at  2-  to  4-kilometer  ranges  is  not  adversely 
affected  by  this  display  field-of-view.  Image  quality  is  excellent  and 
vertical  image  resolution  is  approximately  three  arc  minutes  per  TV  line. 

The  cost  for  two  infinity  mirror/ Beamsplitter  display  systems  would 
have  been  prohibitive  for  the  funding  allowed  under  the  program.  However, 
after  the  completion  of  the  Apollo  lunar  program,  EOSS  personnel  acquired 
several  of  the  Farrand  optical  display  systems  used  in  the  Apollo  simulators. 
The  optics  were  transferred  to  The  University  of  Alabama  and  cut  to  form 
spherical  mirrors  and  beamsplitters  for  the  windscreen  displays.  Two  new 
mounts  for  the  mirrors,  beamsplitters  and  21-inch  Sony  color  television 
systems  were  designed,  built  and  assembled  by  the  University's  personnel. 

The  cockpits  can  be  used  in  at  least  three  modes  of  operation: 

(1)  pilot  flying  gunship  and  handling  fire-control  of  weapons-delivery  with 
copilot/gunner  merely  serving  as  an  observer;  2)  pilot  flying  gunship  and 
copilot/gunner  operating  fire-control  units  such  as  would  be  required  in  a 
man-in-the-loop  missile  flight;  and  3)  as  a  subject/experimenter  system  with 
the  subject  occupying  the  pilot's  cockpit  and  the  experimenter  occupying  the 
copilot  cockpit. 

EOSS 


The  6-DOF  EOSS  described  in  Ref.  1  generates  the  realistic  and  pre¬ 
cisely  controlled  spectral  and  dynamic  environments  required  for  the  eval¬ 
uation  of  optically  guided  systems.  Five  degrees  of  freedom  (Figure  3)  are 
provided  by  a  vertical  assembly  system  consisting  of  a  three-axis  gimbaled 
camera/seeker  mount  which  can  be  moved  laterally  along  a  beam  which  in  turn 
can  be  driven  vertically  between  two  columns.  A  large,  moving  three- 
dimensional  terrain  model  (scaled  1/600)  provides  the  sixth  degree  of  free¬ 
dom  which  simulates  range  closure.  An  AD/4  analog  computer  provides  input 
commands  for  the  position,  rates  and  accelerations  of  each  of  the  six  motion 
drives  and.  In  addition,  completes  the  Euler  angle  transformation.  The 
newly  added  cockpits,  displays,  and  VDS  units  provide  the  EOSS  with  a  man/ 
machine/target  interface  and  the  capability  for  supplying  a  total  weapons 
system  simulation. 

Helicopter  Simulation 

The  requirements  for  the  EOSS  helicopter  simulation  model  are  as 

follows: 


1.  A  simplified  mathematical  force  and  moment  model  of  a  generalized 
helicopter  for  piloted  simulation.  The  simulation  equations  of  motion  must 
be  implemented  on  the  EOSS  AD/4  analog  computer. 


2.  The  helicopter  model  should  provide  good  flight  dynamics  simulation 
fidelity  over  the  0-  to  100-knot  speed  range. 

3.  The  equations  must  be  in  a  generalized  form  so  that  coefficients 
can  be  chosen  to  represent  a  broad  range  of  vehicles  with  minimum 
modification. 

A.  The  rotor  model  should  be  general  enough  to  provide  for  implemen¬ 
tation  of  various  autopilot  and  HUD  display  concepts  for  air-to-ground 
weapons  system  simulation. 

The  helicopter  model  selected  for  the  EOSS  implementation  was  developed 
by  NASA  Ames  Research  Center  personnel  (Reference  2)  for  flight  dynamics, 
terminal  area  guidance,  and  avionics  systems  studies.  Developed  primarily 
as  a  UH-1  simulation,  the  rotor  equations  are  sufficiently  general  so  that  a 
wide  range  of  helicopter  configurations  can  be  modeled.  NASA  Ames  Research 
Center  used  an  all-digital  real-time  implementation  of  the  subject  equations 
to  support  the  broad  range  of  user  applications. 

Considerable  simplification  was  required  to  implement  the  NASA  simula¬ 
tion  on  the  EOSS  AD/A.  Simulation  hardware  verification  and  simplification 
was  accomplished  with  the  aid  of  a  PDP11/ 3A-based  numerical  integration  of 
the  equations  of  motion  reported  in  Reference  2.  Additionally  the  simplified 
equations  implemented  on  the  analog  computer  were  numerically  integrated  for 
comparison  with  the  reference  case.  Comparisons  were  made  at  100  ft/sec, 

50  ft/sec,  and  20  ft/sec.  Simplifications  were  accepted  as  valid  if  the 
dynamic  response  error  for  control  inputs  were  small. 

The  nonlinear  helicopter  simulation  uses  approximately  A0  multipliers 
and  120  amplifiers.  Resolvers  are  used  for  Euler  angle  transformations. 

Currently  the  EOSS  helicopter  model  is  being  used  to  analyze  helicopter- 
fired,  unguided  missile  dispersion.  Effects  of  helicopter  flight  dynamics 
on  autopilot  configurations  for  weapons  pointing  and  HUD  target  displays  are 
subjects  of  current  research. 

The  model  employs  a  "quasi-static  main  rotor  representation,  uniform 
inflow  over  the  rotor  disk,  and  simple  expressions  for  the  contribution  of 
the  tail  rotor,  fuselage,  and  empennage."  Reference  2  presents  a  detailed 
validation  of  the  model  performance  by  comparison  with  UH-1  flight  data. 

In  Reference  2  conclusions  are  stated:  "The  model  appears  to  be  satisfactory 
for  flying-qualities  investigations  at  forward  speeds  and  usable,  but  less 
realistic,  for  hover."  The  model  appears  to  be  quite  acceptable  for  the 
general  class  of  studies  proposed  for  the  WDS  system. 

VPS,  Microprocessor  Video  Display  Systems 

Two  VDS  microprocessors  (See  Figure  A)  are  used  to  generate  an 
out-the-cockpit  display  produced  by  a  21"  raster  scan  color  TV  and  mirror- 
beamsplitter  optics  and  an  instrument  panel  display  that  uses  a  12"  TV  with 
separate  red,  green,  and  blue  inputs.  The  out-the-cockpit  imagery  is  formed 
by  mixing  video  from  a  terrain-board-TV-camera-probe  with  VDS-generated 
special  effects.  The  VDS  uses  dot-mapped  graphics  techniques  to  produce 
such  effects  as  missile  tracers,  target  designators,  and  firing  and  impact 
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flashes  and  can  produce  color  or  black-and-white  data  displays  that  can  be 
overlayed  with  standard  RSI 70  video  from  a  terrain  board  probe.  Each  VDS 
system  can  be  controlled  from  its  own  terminal  or  operate  in  a  slave  mode  to 
a  host  computer  such  as  the  PDP11/34  minicomputer.  Much  of  the  computational 
load  for  graphics  and  special  effects  is  done  by  the  VDS,  freeing  the  host 
computer  for  tasks  such  as  missile  and  aircraft  simulation  with  rapid  refresh 
rates. 

Simulation 


As  the  helicopter  model  runs  on  an  AD/4  hybrid  analog  computer,  the 
PDP11/34  monitors  aircraft  and  missile  parameters  (A/D  channels),  transforms 
stored  inertial  coordinates  to  pilot 's-eye  perspective,  and  sends  screen  co¬ 
ordinates  and  instrument-panel  data  to  the  VDS  microcomputer  through  a  16-bit 
parallel  output  port  (DR  11).  Weapons  system  projectile  tracer  trajectories 
are  computed  by  the  PDP11/34  and  displayed  through  VDS  #1  in  a  real-time 
overlay  of  the  terrain  board  image.  Target  miss-distances  are  automatically 
computed  and  displayed  by  the  host  computer.  Up  to  16  missile  trajectories 
can  be  simultaneously  displayed  to  allow  for  rapid  fire  or  missile  volley 
operation. 

The  PDP11/34  and  VDS  can  transform  a  250-point,  three-color  night¬ 
landing  runway  scene  to  a  pilot' s-view  perspective  image  and  update  a  color 
instrument  panel  with  seven  digital  readings  plus  an  artificial  horizon  at  a 
rate  of  approximately  15  times  a  second. 

The  VDS  dot-mapped  raster  scan  graphics  resolutions  of  320  dot  x  400 
line  interlaced  scan,  black  and  white  or  320  x  200  with  8  colors  are  hardware 
switch-selectable.  The  VDS  uses  a  6502  microprocessor  whose  addressing  modes 
(especially  indexed  indirect  and  indirect  indexed)  and  fast  and  efficient 
memory  access  cycle  make  it  excellent  for  rapid  data  movement. 

Three  8k  x  8  visible  memory  boards  with  cycle  stealing  generated  video 
are  controlled  by  three  bits  for  each  pixel  in  a  320  dot  by  200  line  format. 
Stock  Micro  Technology  Ltd.  memory  boards  are  modified  to  operate  as  slaves 
to  a  master  video  sync  generation  board. 

Graphics  Software 

The  VDS  which  is  implemented  on  the  PDP11/34  and  which  does  the 
transformation  to  pilot' s-eye  perspective  with  fixed-point  binary  fraction 
arithmetic  and  sine,  cosine  lookup  tables  used  for  fast  coordinate  trans¬ 
formations.  The  PDP11/34  communicates  with  the  VDS  using  a  DR11  parallel 
16-bit  output  port  using  two  sequential  16-bit  words.  When  32  bits  of  data 
are  received  by  the  VDS,  an  interrupt  is  generated  to  initiate  the  graphics 
program  stored  in  VDS  read-only  memory  (ROM).  Eight  of  the  32  bits  are 
designated  as  a  control  byte  through  which  the  PDP  passes  operational  com¬ 
mands  to  the  VDS.  The  VDS  graphics  options  are  designed  to  download  as  much 
computation  from  the  minicomputer  as  possible.  The  interrupt-service  routine 
options  contained  in  microprocessor  ROM  are  as  follows: 


Instrument  Panel 
2x2  Tracer 
17-Point  Best  Fit  Line 


Flash 

Target  Designator 

Execute  VDS  Program  Starting  at  VMA 


BIO  =  1  Fill  Screen  Inverted  (dark)  ASCII  Char,  at  VMA 

BIO  =  0  Erase  8K  Block  ASCII  Character  at  VMA 

B9  =1  Red  VM  Erase  dot  at  VMA 

B8  =1  Blue  VM  Erase  Word  at  VMA 

B8  =  B9  =  0  Green  VM  Transfer  Byte  of  Data  to  VMA 

Display  or  erase  two  adjacent  dots  Display  Single  Dot 

(BIO  =  1  erase,  BIO  =  0  write)  Tracer  zones  with  autoerase 
Vertical  Block  Fills  8  dots  wide  option  (b7  =  1) 

by  no.  of  lines  =  (PORT+2) 

To  obtain  speed,  a  number  of  different  interrupt -service  routines  are 
used,  each  written  to  efficiently  handle  a  particular  task.  With  an  eight- 
bit  control  word,  256  options  could  be  included  in  one  interrupt-service 
routine,  but  the  program  overhead  to  test  these  options  would  make  the 
average  interrupt  execution  time  prohibitively  long.  Therefore,  interrupt 
service  routines  with  a  limited  number  of  options  were  designed  for  each  of 
the  different  simulation  tasks  (tracers,  instrument  panel,  windscreen  view, 
color  graphics). 

Air-to-Ground  Weapons  Delivery  Scenario 

A  typical  scenario  simulated  in  the  projectile  display  mode  is  a 
pilot-weapons-system-operator  flying  a  helicopter  missile-carrier.  The 
analog  simulation  of  the  aircraft  on  the  AD/4  analog  computer  receives  pilot 
control  inputs  from  the  simulator  cockpit  and  outputs  aircraft  inertial  po¬ 
sition  and  Euler  attitude  angles.  These  outputs  are  used  to  drive  the  EOSS 
6-DOF  drives  such  that  the  flight  table  TV  camera  which  provides  the  terrain 
visual  scene  is  at  the  proper  position  and  attitude  over  the  terrain  model. 
The  operator  can  fire  a  missile  or  projectile  (also  simulated  on  the  analog 
or  digital  computer)  at  a  target  whose  position  on  the  terrain  model  is  de¬ 
fined  by  a  surveyed  target  data  base.  The  relative  missile-firing  point 
inertial  position  and  missile  attitude  angles  are  output  from  the  simulation. 
The  aircraft  kinematics  outputs  are  used  to  drive  the  flight  table  mounted 
TV  camera  viewing  the  terrain  model  and  along  with  the  missile  kinematics 
provide  graphics  display  information  to  the  PDP11/34  analog-to-digital  con¬ 
verters.  The  PDP11/34  samples  the  aircraft  and  missile  kinematics  inputs 
approximately  20  times  per  second  and  repetitively  computes  the  pilot  per¬ 
spective  view  imagery  of  the  missile  tracer  as  seen  by  the  pilot.  The 
PDP11/34  computes  the  TV  screen  (pilot  outside-the-window  image  plane)  co¬ 
ordinates  of  the  missile  tracer  at  a  particular  instant  of  time  and  passes 
the  data  to  the  VDS  approximately  twenty  times  per  second. 

The  VDS  interprets  the  data  received  and  writes  into  one  bit  of  display 
memory  to  display  a  dot  on  the  TV  screen  at  the  pilot  image  perspective  view 
missile  location  for  the  current  time.  The  VDS  automatically  remembers  the 
dot  location  in  memory  and  erases  it  when  the  next  missile  position  becomes 
available.  The  net  effect  is  to  display  a  missile  tracer  or  flare  as  a 
series  intermittent  of  dots  along  the  computed  missile  trajectory.  The  dis¬ 
play  is  used  to  create  the  illusion  (simulation)  of  a  moving  missile  tail 
flare  or  projectile  tracer,  thus  providing  the  operator  with  targeting 
information. 

The  VDS  PDP11  digital  software  has  the  capability  of  trajectory- 
display  generation  for  single  missile  tracers  or  multiple  rapid-fire  pro¬ 
jectile  tracers.  Single-missile  tracer  trajectory  information  is  generated 
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externally  to  the  PDP11  via  the  analog  computer  solution  and  is  monitored  bv 
the  VDS  software  analoe-to-diaital  routines.  Rapid-fire  tracer  tralectorv 
software  for  the  PDPll  can  handle  computations  for  8  prolectiles  simul¬ 
taneously.  Rapid-fire  VDS  software  utilizes  a  tralectorv  lookup  table  for 
reference  proiectile-tralectorv  generation  and  internally  performs  all  dis¬ 
play  perspective  view  computations  necessary  for  display  generation. 


The  missile-trajectory  is  viewed  by  the  operator  as  a  point  relative  to 
the  aircraft  windscreen  (pilot  f ield-of-view  if  the  pilot  is  looking  parallel 
to  the  aircraft  longitudinal  axis)  or  as  a  perspective  point  superimposed 
over  a  target  image  as  seen  by  a  TV  camera  viewing  the  target  scene. 

Figure  5  illustrates  the  viewing  geometry. 


The  geometry  and  reference  frames  used  for  single-projectile  trajectory 
computation  are  illustrated  in  Figure  6.  The  x^y^z^  axes  are  used  as  an 

earth-fixed  reference  for  aircraft  position  or  fixed-platform  location, 
x^y^z^  are  aircraft  body  axes  with  Euler  attitude  angles  defined  by  a  ifi,  0,  <p 

Euler  angle  sequence.  The  x.y  z.  origin  is  located  at  the  pilot 's-eye  point 

AAA 

and  x^  is  parallel  to  the  aircraft  longitudinal  axis.  The  missile  is 
launched  from  an  aircraft  parallel  to  the  x^  axis  at  time  t^.  and  is  aimed  by 
the  pilot  maneuvering  the  aircraft  to  align  the  x^  axis  so  that  its  extension 
passes  through  the  target.  The  missile  wind-axis  system  x  y  z  is  assumed  to 

WWW 

be  initially  (t  )  aligned  with  the  aircraft  x  y  z  body  axes  so  that 
i  AAA 


*A  =  V 


0A 


'W 


During  missile  flight,  the  relative  missile  aircraft  trajectory  is  com¬ 
puted  referenced  to  x^y^z^  (0)  axes,  transformed  point  by  point  into  the 

x.y  z.  axis  system,  and  then  normalized  to  create  the  perspective  view  co- 
AAA 

ordinates  seen  by  the  pilot.  The  relative  aircraft  missile  position  is: 


XI  =  XE0  -  XA  +  XE 


yl  =  yE0  ‘  yA  +  yE 


Z,  =  Z-„  -  z.  +  z_ 
I  E0  A  E 


where 


x_,  y_,  •  relative  missile/firing-point  positon 

Ebb 

XEQ»  yE0(  ZE0  “  firing-point  position 
x^,  y^,  z^  *  aircraft  position,  etc. 
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FIGURE  5 


OPERATOR  VIEWING  GEOMETRY 


where  160  is  the  horizontal  offset  from  the  center  of  the  TV  screen  and  100 
is  the  vertical  offset.  K15  and  K16  are  scaling  factors  which  are  dependent 
on  the  TV  camera  FOV.  VY  and  VZ  are  the  TV  screen  dot  matrix  locations  and 
VY  ranges  from  0  to  320  and  VZ  ranges  from  0  -  200  as  shown  in  Figure  7 . 


VY 

>• 


320,  200 


VZ 


Figure  7.  TV  Screen,  Graphics  Overlay  Coordinates 


The  computer  graphics  overlay  uses  the  dot  reference  coordinates  VY,  VZ. 

The  constants  K15  and  K16  effectively  scale  the  display  f ield-of-view. 

III.  SUMMARY  AND  CONCLUSIONS 

Through  the  use  of  salvageable  equipments  from  other  government 
programs  and  the  ingenious  application  of  microprocessor  technology,  the 
Army  Missile  Command  has  been  able  to  expand  its  utility  of  its  hardware- 
in-the-loop  EOSS  facility  to  include  man-in-the-loop  capability.  Experi¬ 
mental  studies  of  the  man/machine/missile  interactions  can  now  be  per¬ 
formed  which  should  greatly  improve  the  Army’s  ability  to  resolve  many 
problems  in  the  laboratory  which  previously  had  to  be  done  in  the  field  or 
piecemeal  in  the  laboratory. 

The  hardware  has  been  installed  and  the  software  debugged.  The  system 
is  being  checked  out  preparatory  to  running  the  first  experimental  studies 
on  an  air-to-ground  weapons  delivery  study.  Although  this  low-cost  VDS 
simulator  has  been  developed  primarily  to  study  the  problems  of  a  gunship 
pilot  launching  high-velocity  kinetic-energy  unguided  projectiles,  the 
utility  of  the  WDS  is  foreseen  to  be  useful  in  many  applications.  Problems 
associated  with  launching  and  guiding  systems  such  as  TOW,  HELLFIRE  and 
conventional  ammunition  are  typical  applications.  Human-factor  studies  of 
task-loading  of  pilot  and  copilot/gunner  constitute  another  category  of 
applications,  and  the  adaptability  of  additional  components  such  as  HUDs  and 
helmet-mounted  sights  will  extend  the  range  of  this  versatile  simulation 
facility. 
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The  Influence  of  Figural  Complexity  on  the  Detection,  Recognition,  and 
Identification  of  Targets  in  CGI  Displays 

Abstract 

The  aim  of  this  study  was  to  determine  the  minimum  amount 
of  complexity  necessary  in  a  computer  generated  scene  for  the  detec¬ 
tion,  recognition,  and  identification  of  targets.  Complexity  was 
manipulated  by  modeling  six  targets  in  full  detail  and  then  removing 
detail  until  there  was  just  enough  to  model  a  three-dimensional  object. 
The  targets  were  embedded  in  a  computer-generated  scene,  photographed, 
and  made  into  slides,  The  slides  were  shown  to  three  groups  of  sub¬ 
jects  in  three  experiments.  In  the  first,  subjects  were  asked  to 
detect  the  presence  of  a  target;  in  the  second,  they  were  asked  to 
recognize  which  of  six  possible  targets  it  was  and  in  the  third, 
they  were  asked  to  identify  which  of  four  possible  targets  of  the 
same  type  this  particular  target  was.  Target  complexity  had  no  sig¬ 
nificant  effect  on  detection  or  recognition.  However,  more  complex 
targets  were  much  easier  to  identify,  especially  when  complexity  was 
in  the  form  of  internal  detail.  Identification  errors  greatly  exceed¬ 
ed  errors  in  recognition,  and  errors  in  detection  were  relatively  rare. 

Introduction 

Low  level  night  flying  of  high  performance  aircraft  depends 
for  its  success  on  the  amount  and  interpret ability  of  information 
about  the  terrain  below  and  in  front.  Direct  visual  information  may 
be  severely , limited  and  night  sensing  devices  such  as  Low  Level  Tele¬ 
vision  (L3TV)  and  Forward  Looking  Infra-Red  (FLIR)  are  essential 
aids  In  many  missions. 

These  electrooptical  (E/0)  displays,  now  installed  in  some 
aircraft,  require  training  in  their  use  and  computer  generated  image 
(CGI)  simulations  have  been  developed  for  this  purpose.  The  level  of 
transfer  of  training  in  using  such  images  has  been  assumed  to  be  a 
function  of  realism  which  in  turn  is  seen  to  be  related  to  scene  com¬ 
plexity.  Complexity  in  this  context  is  defined  as  the  addition  of 
features  to  the  display,  each  of  which  Increases  the  fidelity  to  the 
E/0  sensor  display  by  a  small  amount  at  an  increasing  cost  (Bunker 
A  Heeschen,  1975). 

The  question  then  arises  as  to  the  extent  of  complexity  nec¬ 
essary  to  permit  positive  transfer  of  training  to  the  task  being  simu¬ 
lated.  The  present  study  addresses  itself  to  this  problem. 

CGI  simulations  normally  have  a  "cartoon-like"  quality 
which  is  quite  different  in  appearance  from  the  E/0  sensor  displays. 
Moreover,  the  CGI  pictures  are  made  up  of  a  large  number  of  elements 
or  edges  which  go  to  depict  each  object  in  the  scene.  If  every  de¬ 
tail  of  an  object  is  represented,  an  extremely  large  number  of  edges 
is  required,  with  a  commensurate  cost  increase.  However,  objects  can 
be  represented  in  a  recognizable  way  using  only  a  moderate  number  of 
edges.  The  necessary  number  of  edges  for  the  completion  of  several 
tasks  is  the  subject  of  this  report. 

In  a  previous  study  (LeMay  A  Reed,  1960),  the  addition  of 
visual  noise  and  an  edge  transfer  function  to  the  scene  was  also  in¬ 
vestigated.  The  transfer  function  effect  was  not  significant,  but 
the  affect  of  noise  was,  so  only  noise  was  included  in  the  present 
study.  Since  the  effect  of  number  of  edges  in  a  scene  and  visual 
noise  may  be  expected  to  dopond  on  the  task  being  performed,  throe 

107 


representative  task; a  were  examined  in  the  present  study*  They 
are  the  detection,  recognition,  and  identification  of  objects  in  a 
simulated  scene* 

Detection  is  defined  as  the  ability  of  an  operator  to 
discriminate  between  the  presence  and  absence  of  a  target;  recogni¬ 
tion  is  the  assignment  of  a  detected  target  to  a  class  of  possible 
targets;  and  Identification  is  the  selection  of  one  individual 
target  from  a  class  of  targets* 

Detection 

Studies  on  the  detection  of  just  barely  visible  (or  audible) 
targets  in  a  background  of  noise  have  been  numerous  since  the  incep¬ 
tion  of  the  work  on  the  theory  of  signal  detection  (TSD)  described 
by  Green  and  Swets  (1966).  The  general  problem  of  visual  search  for 
complex  targets  was  reviewed  by  Teighner  and  Moeharnuk  (1979). 

They  defined  complexity  in  terms  of  stimulus  dimensionality,  i.e., 
the  number  of  attributes  such  as  form,  direction,  color,  etc*, 
which  a  stimulus  may  have.  They  reviewed  the  findings  of  nine  earlier 
studies,  and  concluded  that  time  to  detect  a  target  decreases  as  the 
number  of  stimulus  dimensions  (complexity)  increases  and  that  rate 
of  stimulus  processing  increases  (i.e.,  time  per  stimulus  decreases) 
as  number  of  stimulus  dimensions  increases.  In  other  words,  complex 
targets  are  more  easily  detected  and  processed  (recognized?)  than 
simpler  targets. 

Uttal  and  Tucker  (1977)  present  a  multi-stage  model  of 
perception  in  which  each  stage  can  be  explored  only  if  thresholds 
for  previous  stages  have  been  exceeded.  They  measured  performance 
in  a  detection  task  and  found  that  complexity  was  a  powerful  de¬ 
terminant  of  susceptibility  to  masking  by  noise. 

These  and  other  studies  offer  somewhat  contradictory,  and 
not  always  relevant  predictions  for  the  present  study.  It  seems 
that  complexity  sometimes  aids  and  sometimes  hinders  detection  per¬ 
formance.  Most  directly  relevant  is  the  study  by  Uttal  and  Tucker 
(1977),  which  found  that  complexity  hindered  detection,  especially 
in  the  presence  of  noise.  The  present  study  uses  simulations  of 
real-life  targets,  and  subjects  them  to  a  controlled  analysis  in 
the  laboratory.  It  is  expected  that  complexity  will  have 
little  or  no  effect  on  detection  performance,  and  that  the  presence 
of  noise  will  hinder  performance,  perhaps  interacting  with  complexity. 

Recognition 

The  distinction  between  recognition  and  identification 
has  not  always  been  clearly  made  in  the  literature,  and  the  term 
"discrimination,  **  suggested  by  Gibson  (1969)  has  sometimes  been 
used  for  both  tasks.  For  present  purposes,  a  somewhat  arbitrary 
division  has  been  made  between  those  studies  which  ask  subjects 
to  gocoqnlze  one  of  a  class  of  targets,  such  as  letters  of  the 
alphabet,  and  those  which  ask  subjects  to  identify  a  particular 
target  which  has  usually  been  constructed  by  the  experimenter. 

Pasnak  (1971)  used  a  same-different  recognition  task  with 
simple  and  complex  random  polygons.  Overall  errors  were  greater 
for  simple  than  for  complex  figures,  and  subjects  responded  to  the 
whole  contour  for  simple  shapes,  and  the  distinctive  parts  of  the 
outlines  of  more  complex  shapes. 

Staller  and  Sekuler  (1977)  asked  subjects  to  respond  to 
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mirror-image  and  nonmirror-image  stimulus  pairs  in  a  two-choice  re¬ 
sponse,  and  complexity  seeaed  to  influence  the  quality  of  pattern 
processing,  in  that  particular  targets  were  responded  to  with  signifi¬ 
cant  differences  in  reaction  time. 

A  number  of  studies  (Frowein  &  Sanders,  1978;  Nygard,  et  al., 
1964;  Berlchout  &  Philips,  1979;  Guttraan,  Snyder,  Farley  &  Evans,  1979) 
lead  to  the  conclusion  that  recognition  is  more  difficult  than  detec¬ 
tion,  that  simpler  patterns  may  be  harder  to  recognize  than  complex 
ones,  and  that  complexity  influences  the  quality  of  pattern  processing, 
so  that  there  are  differences  among  particular  targets. 

In  the  present  study,  the  stimuli  are  constructed  so  that 
they  are  increasingly  degraded  by  the  subtraction  of  edges,  both  in 
outline  and  internal  detail,  and  this  is  defined  as  decreasing  target 
complexity.  They  are  further  degraded  by  the  addition  of  visual 
noise  and  these  conditions  are  applied  to  six  different  targets.  It 
may  be  expected  that  noise  will  result  in  more  errors  of  recognition, 
some  targets  should  be  harder  to  recognize  than  others,  and  the  simpler, 
more  degraded  targets  should  be  harder  to  recognize. 

Identification 

Studies  of  the  identification  of  patterns  have  frequently 
made  the  distinction  among  detection,  recognition  and  identification. 

Resell  and  Willson  (1973)  defined  the  three  tasks  as  they  are  defined 
here  and  found  that  detection  probability  rises  with  the  S/N  ration 
of  a  video  tube,  i.e.,  noise  lowers  detection  rate.  Snyder  (1973) 
reviewed  a  number  of  studies  on  detection,  recognition,  and  identi¬ 
fication,  and  elucidated  several  measures  for  use  in  studying  them. 

Most  relevant  here  is  the  percent  or  probability  correct,  which  can 
be  a  measure  of  completeness  (number  of  correct  responses  divided  by 
the  number  of  possible  targets)  or  accuracy  (number  of  correct  re¬ 
sponses  divided  by  the  total  number  of  responses,  correct  and  Incorrect). 
The  former  is  the  measure  used  in  the  present  study.  Snyder  also  in¬ 
troduced  the  MTFA  (mean  transfer  function  area)  as  a  measure  of  overall 
image  quality,  and  found  that  it  was  related  to  judged  quality  and 
information  extraction  as  measured  by  responses  to  a  series  of  questions 
about  a  scene. 

On  the  basis  of  these  and  other  studies,  as  well  as  a 
consideration  of  the  tasks  in  the  present  study,  several  expectations 
may  be  advanced.  It  would  seem  that  more  complex  targets  would  be 
more  easily  identified,  since  this  was  an  actual  finding  in  earlier 
studies,  and  because  better  images  led  to  easier  identification  in 
some  studies.  A  variable  in  the  present  study  which  has  not  been 
isolated  in  previous  research  is  that  of  the  internal  details  of  a 
target,  as  exemplified  by  the  surface  details  of  buildings,  such  as 
doors  and  windows.  A  previous  study  (LeMay  and  Reed,  1980)  found 
these  to  be  important  determinants  of  judged  complexity.  Therefore, 
it  might  be  expected  that  targets  that  are  more  complex  in  terms  of 
internal  detail  will  be  more  easily  identified  than  targets  with  sim¬ 
plified  or  missing  internal  details. 

It  would  also  seem  that  the  functions  relating  target  com¬ 
plexity  to  detection,  recognition,  and  identification  should  be  inde¬ 
pendent  of  one  another,  based  on  Swets,  Green,  Getty,  and  Swets  (1978), 
and  that  detection  should  result  in  the  fewest  errors,  while  identification 


109 


should  result  In  the  most  errors. 

Method 

Subjects 

The  subjects  for  the  detection  experiment  were  14  students 
In  an  experimental  psychology  class  at  Montclair  State  College.  Thir¬ 
teen  of  the  same  subjects  also  served  In  the  recognition  experiment. 
Both  sets  of  data  were  collected  in  a  group  setting. 

The  subjects  for  the  identification  experiment  served 
individually.  They  were  eleven  volunteers  from  classes  at  Montclair 
State. 

Stimuli 

The  stimuli  for  all  three  studies  were  composed  of  108 
slides  of  various  scenes  generated  on  a  computer  according  to  the 
methods  described  by  Bunker  and  Heeschen  (1975).  The  scenes  were  of 
six  targets  at  each  of  six  levels  of  complexity  and  three  levels  of 
visual  noise.  They  were  used  in  a  6  x  6  x  3  factoral  design  with 
repeated  measures  on  the  same  subjects  (Kirk,  1968). 

The  six  targets  were  embedded  in  a  scene  as  they  would  be 
likely  to  appear  when  viewed  from  an  aircraft.  The  targets  consisted 
of  a  (n):  aircraft  hangar;  factory;  water  tower;  single  house;  air¬ 
strip;  single  aircraft  on  the  ground.  Only  one  target  appeared  on 
each  slide. 

Each  target  was  presented  at  each  of  six  levels  of  com¬ 
plexity.  Since  a  previous  study  (LeMay  &  Reed,  1980)  had  shown  that 
the  number  of  internal  details  was* an  important  determinant  of  sub¬ 
jective  complexity,  the  levels  were  chosen  so  that  there  were  two 
levels  of  complexity  of  figure  outline,  and  three  of  added  internal 
detail.  These  are  illustrated  for  one  target  (the  factory)  in 
Figures  1  and  2  The  simple  outline  consisted  of  the  barest  minimum 
number  of  edges  to  outline  two  dimensions  of  an  object,  so  that  the 
basic  building  facade  is  a  rectangle.  The  more  complex  outline  consis 
ted  of  the  greatest  detail  in  the  outline  of  that  particular  building 

The  three  levels  of  internal  edges  consisted  of,  first, 
an  outline  with  no  internal  detail  at  all.  In  the  second  level,  the 
doors  and  windows  were  represented  by  blocks  or  rectangles.  At  the 
third  level,  windows  and  doors  were  articulated. 

When  the  most  complex  level  of  internal  and  external  edges 
were  combined,  the  resulting  stimulus  was  a  fairly  realistic  picture 
of  the  object  represented.  The  other  levels  of  complexity  should  be 
thought  of  as  degradations  of  the  representation. 

Since  the  FLIR  and  L3TV  images  are  generally  viewed  in 
conjunction  with  a  considerable  amount  of  visual  noise,  each  of  the 
36  combinations  of  target  and  complexity  was  subjected  to  two  levels 
of  noise,  as  well  as  being  generated  with  a  clear  picture.  This  added 
a  3-factor  variable,  resulting  in  a  3  (noise  level)  x  2  (internal 
edges)  x  3  (external  edges)  x  6  (targets)  factorial  design  with  108 
conditions.  This  was  presented  to  all  subjects  in  a  repeated-measures 
design. 

The  scenes  were  generated  and  shown  on  a  TV  screen,  and 
still  pictures  were  taken  of  them  and  made  into  slides  and  into  still 
pictures  so  that  the  scenes  could  be  presented  to  subjects  easily. 


110 


Figure  1.  Simple  outline  of  t he  "factory"  target  at  three  levels  of  interna] 


igure  2.  Complex  outline  of  the  "factory"  targe 


Procedure 

Por  the  detection  experiment ,  three  slides  were  taken  of 
the  basic  scene  with  no  targets  in  it,  one  for  each  noise  level* 

These  were  interspersed  with  the  slides  on  which  there  was  a  target 
and  presented  to  the  subjects  for  a  very  short  period  each  in  one 
random  series  of  216  slides  on  which  there  was  a  target  50*  of  the 
time* 

Subjects  were  instructed  to  record  on  an  answer  sheet 
whether  or  not  there  was  a  target  in  the  scene,  regardless  of  which 
target  it  was.  Subjects  wets  not  informed  of  target  probability. 

In  the  recognition  experiment,  the  slides  were  presented 
in  a  random  series  of  108.  This  time,  however,  subjects  were  in¬ 
structed  to  mark  on  an  answer  sheet  which  one  of  the  six  possible 
targets  was  on  the  slide  just  shown.  They  were  first  acquainted 
with  all  of  the  targets,  using  the  scene  with  all  six  targets  in  it, 
in  full  detail,  i.e.,  with  the  maximum  number  of  edges. 

Subjects  were  run  individually  in  the  identification  ex¬ 
periment.  First  they  were  acquainted  with  the  appearance  of  the  tar¬ 
gets  using  a  set  of  drawings  mounted  on  the  wall  in  front  of  them. 
These  consisted  of  four  drawings  of  each  target  type,  one  of  which 
was  the  target  in  the  scene,  and  three  of  which  were  of  other,  simi¬ 
lar  targets.  There  were  24  drawings  in  all:  four  hangars,  four 
water  towers,  four  airstrips,  etc.  They  were  visible  to  the  sub¬ 
jects  at  all  times  throughout  the  experiment. 

Then,  subjects  were  told  they  would  be  shown  a  set  of 
slides  in  which  one  of  the  six  pictured  targets  would  appear.  The 
slides  were  shown  briefly,  as  before,  and  subjects  informed  the  ex¬ 
perimenter  which  target  of  the  24  pictured  was  in  the  slide.  If 
the  subject  was  uncertain,  the  slide  could  be  shown  again,  as  many 
times  as  the  subject  wished,  although  each  time  was  of  the  same 
very  short  duration.  Thus,  two  measures  of  subject  performance  were 
obtained:  first,  number  of  errors  made  in  identifying  the  target 
and  second,  number  of  repetitions  necessary  for  identification. 

A  short  de-briefing  session  was  held  after  each  experiment. 
Subjects  were  informed  of  the  target  probability  in  the  detection 
experiment,  and  had  reached  the  conclusion  that  it  was  .50  before 
the  experimenter  so  informed  them.  In  the  identification  experiment, 
no  subjects  had  been  aware  of  the  fact  that  there  were  only  six 
possible  targets;  they  all  thought  there  had  been  more  than  one 
of  each  target  type. 

Results 

The  dependent  variable  for  the  detection  and  recognition 
experiments  was  whether  or  not  the  subject  had  made  an  error  in 
each  category.  An  error  was  scored  as  one  and  a  correct  response 
as  two.  This  was  also  used  for  identification  and,  in  addition, 
the  number  of  repetitions  of  each  stimulus  slide  was  scored.  These 
data  were  subjected  to  a  treatments  by  subjects  analysis  of  vari¬ 
ance  (Kirk,  1968)  to  determine  the  effects  of  noise,  target  com¬ 
plexity,  and  target  type  on  errors  and,  for  identification,  number 
of  repetlons. 

Detection 

For  the  detection  experiment,  the  only  significant  main 
effect  was  that  of  target  type.  Significant  interactions  were 
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obtained  between  target  type  and  noiset  external  edges  and  internal 
edges,  and  external  edges  and  noise.  The  house,  the  water  tower, 
and  the  factory  were  difficult  to  detect,  and  this  was  especially 
so  when  noise  was  added  to  the  display.  It  should  be  noted  that 
these  are  relatively  small  targets. 

The  interaction  between  noise  and  external  edges  is 
accounted  for  by  a  very  slight  increase  in  difficulty  of  detection 
at  both  higher  complexity  and  higher  noise  levels.  In  other  words, 
the  more  complex  a  target  is  under  noisy  conditions,  the  harder  it 
is  to  detect. 

It  should  be  borne  in  mind  that,  despite  some  significant 
interactions,  the  ability  of  subjects  to  detect  targets  is  not 
really  affected  by  target  complexity,  or  even  much  by  noise.  The 
actual  overall  error  rate  in  the  experiment  was  quite  low.  Sub¬ 
jects  detected  91.88%  of  all  the  targets  presented.  This  result 
is  presented  in  Figure  3  where  it  can  be  compared  with  the  data  ob¬ 
tained  for  recognition  and  identification.  The  overall  false  alarm 
rate  was  18.33%. 

Recognition 

The  results  of  the  recognition  experiment  were  similar  to 
those  for  the  detection  experiment,  in  that  the  only  significant 
main  effect  was  that  for  target  type,  and  there  was  a  significant 
interaction  between  target  type  and  external  edges.  No  other 
effects  were  significant.  This  seems  to  be  the  result  of  the  data 
for  the  plane,  where  more  external  edges  mate  it  more  difficult  to 
recognize,  and  the  factory,  where  the  opposite  is  true.  The  rea¬ 
sons  for  this  are  mentioned  later.  The  plane,  the  hangar  and  the 
factory  were  the  most  difficult  targets  to  recognize. 

The  actual  overall  error  rate  for  recognition  was  consider¬ 
ably  higher  than  that  for  detection.  Only  77.24%  of  all  targets 
presented  were  recognized  correctly.  Although  the  number  of  errors 
declined  as  complexity  increased,  this  result  was  not  significant. 

It  must  therefore  be  concluded  that  increasing  target  complexity 
does  not  enhance  recogniz ability. 

Identification 

The  results  for  identification  are  quite  different  from 
those  for  detection  and  recognition.  Here,  all  main  effects  except 
that  of  noise  are  significant,  and  there  are  no  significant  interactions. 

The  variable  of  most  interest  in  the  present  study  is  that 
of  complexity.  Both  internal  and  external  edges  had  a  significant 
effect  on  errors  of  identification  (internal:  F-l6.83|p<*00i; 
external:  F«6.13j p<.03).  These  results  are  also  presented  in  Figure  3 
where  it  is  clear  that  targets  with  more  detailed,  complex  outlines 
are  more  easily  identified  than  those  in  which  the  outline  has  been 
degraded  by  removal  of  some  detail. 

Degrading  the  target  image  by  removal  of  internal  detail 
has  an  even  more  drastic  effect  on  identification.  The  complete 
absence  of  internal  detail  represented  by  the  least  complex  targets 
results  in  an  80%  error  rate  in  identification.  A  Duncan  multiple 
range  test  indicates  a  significant  difference  between  the  first  and 
second  level  of  complexity,  but  not  between  the  second  and  third 
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Complexity  level  (mean  number  of  edges  used  to 
generate  a  scene) 


Figure  3»  Percentage  of  errors  in  detection,  recognition,  and 
identification  for  six  complexity  levels. 


levels,  i.e.,  absence  of  Internal  detail  makes  a  target  difficult 
to  identify,  but  simply  presenting  some  internal  detail  decreases  the 
error  rate  significantly.  Further  articulation  of  detail  does  not 
result  .in  additional  improvement. 

The  effect  of  target  type  on  identification  errors  was  also 
significant.  This  data  is  presented  in  Table  1  .  The  factors  which 
determine  difficulty  in  particular  target  identification  are  not  clear, 
and  will  bear  further  investigation.  The  overall  error  rate  for  tar¬ 
get  identification  was  59%. 


Table  1 

Mean  error  score  (an  error  was  scored  as  1,  a  correct  identification 
was  2)  for  each  target,  and  mean  number  of  repetitions  for  each  target 
as  a  function  of  noise  level  in  the  identification  experiment. 


Errors 

Repetitions 

No 

Moderate 

Heavy 

Target  Type 

Noise 

Noise 

Noise 

House 

1.78 

2.83 

2.71 

3.45 

Plane 

1.73 

2.26 

2.12 

2.28 

Factory 

1.61 

2.27 

2.04 

2.82 

Hangar 

1.57 

2.54 

2.76 

2.12 

Airstrip 

1.39 

1.92 

1.97 

2.01 

Watertower 

1.  38 

2.08 

2.16 

2.74 

The  repeated  presentations,  constituted  a  second  dependent 
variable  which  was  subjected  to  the  same  type  of  analysis  of  variance. 
Noise  and  tarfets  were  the  only  significant  main  effects,  and  they 
interacted  significantly  with  one  another.  There  was  also  a  signi¬ 
ficant  interaction  between  external  edges  and  targets. 

Discussion 

The  most  significant  result  of  the  present  investigation 
from  the  point  of  view  of  the  simulation  of  E/0  displays  is  the  find¬ 
ing  that  the  complexity  of  a  target,  in  terms  of  the  number  of  external 
and  internal  edges  which  go  to  make  it  up,  influences  performance  on 
an  identification  task  but  has  no  effect  on  the  recognition  and  de¬ 
tection  tasks  used  here.  Thus,  a  high  degree  of  complexity  in  sim¬ 
ulation  may  be  necessary  only  for  training  operators  in  the  identifi¬ 
cation  of  particular  targets  and  not  for  simply  recognizing  a  class 
of  possible  targets  or  detecting  the  presence  of  a  target. 

Considering  the  similarity  of  the  experimental  situations, 
the  tasks  produced  markedly  dissimilar  results  in  terms  of  the  number 
of  errors.  The  overall  error  rate  was  only  8%  for  the  detection  task. 
It  jumps  to  23%  for  the  recognition  task,  and  to  59%  for  the  identi¬ 
fication  task.  Where  they  are  affected  by  degradation  in  target  com¬ 
plexity,  identification  errors  jump  to  80%.  Detection  is  a  relatively 
easy  task,  and  is  not  affected  by  complexity. 

It  is  not  surprising  that  complexity  did  not  enhance  target 
detectability.  In  fact,  the  addition  of  detail  to  a  target  may,  in 
a  sense,  add  to  a  visual  noise  factor  and  thereby  make  it  harder  to 
detect.  Evidence  for  a  phenomenon  like  this  was  suggested  by  Teichner 
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and  Mocharnuk  (1979)  and  Uttal  and  Tucker  (1977).  The  surprising 
result  here  is  that  the  addition  of  visual  noise  to  the  scene  did  not 
affect  detectability.  It  is  even  more  surprising  that  there  was  no 
interaction  between  noise  and  internal  edges t  since  the  addition  of 
internal  edges  to  a  target  in  an  already  noise  scene  would  seem  to 
make  it  more  easily  seen  as  part  of  the  noisy  background  and  thus 
harder  to  detect,  as  in  the  studies  mentioned.  Instead,  however, 
there  is  an  interaction  between  noise  and  external  edges  and  noise 
and  target  type.  This  is  probably  because  targets  with  some  detail 
in  their  outlines  (the  house  and  the  factory)  were  the  most  difficult 
to  detect  under  noisy  conditions.  These  results  seem  to  indicate 
that  target  complexity  may  have  a  slight  negative  effect  on  detect¬ 
ability,  but  it  is  too  small  to  have  produced  a  main  effect. 

While  it  was  to  be  expected  that  target  complexity  would 
not  effect  detectability,  the  lack  of  any  main  effects  other  than 
that  of  target  type  is  surprising  for  the  recognition  experiment. 

In  particular,  it  would  seem  that  complexity  of  detail  should  affect 
the  recognizability  of  a  target,  especially  since  several  of  the  tar¬ 
gets  were  chosen  because  of  their  rather  unique  shapes,  such  as  the 
hangar,  with  the  distinctively  curved  roof,  or  the  airstrip  with  its 
rather  characteristic  projections  in  various  directions.  Both  of 
these  characteristics  were  missing  in  the  simplified  outlines.  Yet 
neither  target  wes  particularly  difficult  to  recognize,  and  the 
factory  was  actually  easier  to  recognize  without  the  high  level  of  ex¬ 
ternal  detail.  If,  as  Gibson  (1969)  suggests,  pattern  recognition  de¬ 
pends  on  the  presence  of  "critical  features,"  it  is  not  evident  in 
the  recognition  part  of  these  experiments.  It  would  seem  that  there 
is  enough  variability  in  the  shapes  of  the  objects  chosen  so  that  a 
fairly  large  degree  of  departure  from  either  the  outline  or  the  in¬ 
ternal  features  of  a  particular  target  does  not  render  it  unrecog¬ 
nizable  as  a  member  of  a  certain  class  of  targets. 

While  the  absence  of  detail  does  not  seem  to  affect  target 
recognition,  it  has  a  very  clear  effect  on  the  exact  identification 
of  a  target  as  being  one,  and  only  one,  of  its  general  class.  The 
application  of  Gibson's  (1969)  critical  feature  model  would  seem  to 
be  relevant  here.  Both  external  and  Internal  edges  had  a  significant 
effect  on  identification.  Degrading  a  target  image  by  subtracting 
either  external  or  internal  critical  features  makes  it  more  difficult 
to  Identify. 

This  is  perhaps  most  readily  apparent  in  the  case  of  in¬ 
ternal  edges.  The  total  absence  of  any  internal  edges,  as  in  the 
most  simple  targets,  results  in  an  extremely  high  error  rate  (over  70% 
even  for  targets  with  a  complex  outline).  Adding  any  amount  of 
internal  detail  immediately  brings  the  error  rate  down  near  50%  for 
both  simple  and  complex  outlines.  Internal  details  certainly  seem 
to  act  like  critical  features  for  the  identification  of  targets. 

This  observation  is  borne  out  by  the  significance  of  the  difference 
between  the  first  and  second  levels  of  internal  detail  (the  presence 
or  absence  of  any  detail)  and  tho  lack  of  a  significant  difference 
between  the  second  and  third  levels. 

The  absence  of  an  interaction  between  external  and  internal 
edges  is  reflected  in  the  relationship  shown  in  Figure  3  The  effect 
of  internal  edges  with  a  complex  outline  is  repeated  for  the  simple 
outline,  but  at  a  lower  error  rate.  Thus,  internal  and  external 
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detail  act  separately  and  additively  to  lower  the  error  rate  for 
target  identification.  There  are  no  ether  significant  interactions, 
either  with  noise  or  with  target  type,  indicating  that  the  main 
effects  of  complexity  apply  across  the  various  types  of  targets  and 
levels  of  noise.  In  other  words,  where  complexity  is  an  important 
variable,  as  in  the  identification  task,  it  acts  by  itself  in  a  simple 
fashion  regardless  of  other  variables.  This  should  make  it  easier 
to  deal  with  in  an  applied  situation. 

The  measure  of  number  of  repetitions  was  not  significantly 
affected  by  complexity,  but  only  by  target  type  and  noise.  It  is 
not  surprising  that  some  targets  require  more  time  to  identify,  or 
that  these  targets  should  be  the  most  difficult  to  identify  in  terms 
of  errors.  The  effect  of  noise  is  also  to  be  expected,  since  noise 
increases  uncertainty.  This  would  probably  be  true  even  if  the  error 
rate  were  not  related  to  the  number  of  repetitions,  i.e.,  even  if 
they  had  guessed  correctly,  subjects  would  still  prefer  more  time 
to  examine  a  target  under  high  noise  then  under  low  noise  conditions 
(Swets,  Green,  Getty  &  Swets,  1978). 

It  is  interesting  to  note  that,  when  subjects  were  ques¬ 
tioned  after  participating  in  the  identification  experiment,  none 
of  them  realized  that  there  were  only  six  targets,  with  six  different 
versions  of  each.  They  all  thought  that  there  was  more  than  one  air¬ 
strip,  factory,  house,  etc.  This,  of  course,  reflects  the  high 
error  rate,  but  it  also  emphasizes  the  importance  of  the  complexity 
of  figures  for  the  identification  process. 

Conclusions  and  Recommendations 

1.  Since  hits  and  false  alarms  in  detection,  and  errors  in 
recognition,  are  not  affected  by  complexity  either  in  external  or 
internal  detail,  a  very  low  level  of  such  detail  is  probably  all 
that  is  necessary  for  the  simulation  of  these  tasks.  Bare  outline 
figures,  such  as  those  used  at  the  simplest  level  in  the  present 
study,  are  probably  sufficient  in  CIG  simulation. 

2.  Errors  in  identification  are  drastically  affected  by  com¬ 
plexity,  and  it  is  probably  necessary  to  simulate  considerable  detail 
if  unacceptable  levels  (70-80%)  of  error  are  to  be  avoided.  The 
best  combination  for  use  in  identification  training  is  probably  the 
higher  level  of  outline  complexity  with  the  middle  level  of  internal 
complexity,  since  the  third  level  of  internal  complexity  does  not 
seem  to  lower  the  error  rate. 

3.  Error  rates  for  recognition  and  identification  are  high, 
and  may  be  unacceptable  for  practical  application.  They  may  be 
improved,  however,  with  training,  and  training  may  interact  with 
complexity,  so  that  subjects  may  be  trained  to  recognize  and  identify 
simpler  targets.  More  research  is  needed  in  this  area. 

4.  Noise  is  an  important  variable  which  has  significant  effects 
on  all  three  tasks,  and  interacts  with  target  characteristics  which 
are  not  covered  by  the  manipulation  of  target  edges,  as  is  shown  by 


the  interaction  of  noise  with  target  type*  The  role  of  noise 
should  be  further  investigated,  (especially  as  it  pertains  to 
the  simulation  of  E/0  displays*) 
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ABSTRACT 


Hitachi  Denshi  Co.,  Ltd.  has  been  manufacturing  trainers,  simulators  and  visual 
display  systems  for  more  than  20  years.  All  of  their  systems  have  been  sold 
only  in  Japan,  to  the  Japan  Defense  Agency,  universities  and  domestic  industry. 

A  short  history  of  the  company,  a  partial  list  of  some  of  the  visual  systems, 
trainers  and  simulators  has  been  included  plus  a  general  outline  of  the  full 
color  -  daylight  CGI  display  system  Hitachi  is  marketing  today. 


INTRODUCTION 


Hitachi  Denshi  Co.,  Ltd.  has  more  than  20  years’  experience  in  the  research, 
development  and  manufacture  of  flight  and  other  simulators  and  visual  display 
systems  for  a  variety  of  simulators  and  other  training  applications. 

They  are  currently  able  to  meet  the  visual  display  requirements  of  the  new  FAA 
Phase  II,  with  their  "Bestview  series",  and  will  shortly  be  able  to  deliver  a 
Phase  III  CGI  visual  display  system  with  their  full  natural  color  -  cockpit 
bright/daylight  pictures  and  their  hybrid  modelling  techniques. 

Over  the  last  few  years  Hitachi  Denshi  has  acquired  world-wide  recognition 
through  highly  developed  systems  technology  in  broadcasting  systems,  CCTV 
systems,  radio  communications  systems,  information  processing  systems,  oscillo¬ 
scopes,  integrated  circuits,  image  orthicons,  image  processing  systems,  photo¬ 
multiplier  tubes,  tri-electrode  color  videcon  tubes  and  color  freeze  picture 
transmission  systems,  etc.  They  are  looking  forward  to  the  time  that  industry 
and  the  defense  services  will  recognize  the  quality  and  the  uniqueness  of  their 
image  systems. 

Hitachi  Denshi  Co.,  Ltd.  is  a  division  of  the  Hitachi  Ltd.  group  which  was 
founded  in  1910.  Hitachi  Ltd's  net  sales  in  1979  eas  over  12  billion  US 
Dollors.  They  employ  about  150,000  people  in  40  main  companies  and  about  440 
subsidiaries. 
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Bestview  Major  Specifications 
Item 


Specif ications 


Visual  Display  Capacities 
Position  Resolution 
Available  Hues 


Colors  available 


Scene  Update  Rate 


bits 


Display  Surfaces  350  * 

Display  Light  Points  4,000  * 

Vertical  483 

Horizontal  1,024 

Red  6 

Blue  6  " 

Green  6  " 

All  natural  colors  are  available  by 
the  above  color  element  combinations. 
Surface  49  colors 

Light  Point  15  " 

Total:  64  colors 

30  Hz,  interlace  television  system 


Occultation 


10  levels 


Edge  Smoothing 

Numbers  of  Channel  and 
Display  Units  (windows) 


Yes 

1  channel  and  2  windows  to 
4  channel  and  6  windows 


Brightness 
CRT  Resolution 


Deflection  Distortion 


More  than  30  ft-lm  reference  white 
brightness  on  CRT  surface 
Vertical  more  than  350 

Horizontal  "  500 

(measured  by  monoscope  signal) 

Less  than  7% 


Raster  Distortion 


Less  than  2 %  (BTS  method) 


Display  Units 


26"  RGB  shadow  mask  CRT  -  of  very  high 
brightness,  sharp  pictures. 


*  The  350  surfaces  and  the  4,000  light  points  are  generated  separately 
and  can  be  displayed  separately  or  together  for  more  scene  flexibility. 


Bestview  Field  of  View  Options 


Radius  (mm) 

Horizontal 

Vertical 

Type 

1 

1,200 

47  degrees 
or  more 

36  degrees 
or  more 

Type 

2 

1,300 

44  degrees 
or  more 

33  degrees 
or  more 

Type 

3 

1,400 

40  degrees 
or  more 

30  degrees 
or  more 

Actual  field  of  view  is  determined  by  the  dimensions  of  the  cockpit. 
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GENERAL  DESCRIPTION  OF  HITACHI' A  BESTVIEW  VISUAL  DISPLAY  STSTEMS 


The  computer  generated  image  system  has  been  used  by  Hitachi  in  many  fields  of 
computer  technology  besides  in  their  visual  display  system  for  simulators. 

Hitachi  has  progressed  by  continuous  research  and  development  for  more  accurate 
scene  generations  and  as  a  result  of  this  expects  to  have  even  greater  image 
realism  very  soon. 

As  early  as  in  1975,  Hitachi  Denshi's  research  and  development  of  computer 
generated  images  were  much  further  advanced  than  many  of  their  competitors' 
visual  systems  are  today. 

The  Bestview  system  is  a  CGI  display  system  developed  to  provide  full  natural 
color  daylight  along  with  twilight,  dusk  and  night  scenes  to  give  the  pilots 
as  much  clarity  and  realism  as  possible. 

The  visual  display  system  operates  as  follows: 

1)  Due  to  the  raster  scan  operation  as  against  the  caligraphic  operation,  full 
natural  daylight  colors  can  be  presented  which  meet  the  requirements  today 
of  the  later  phases  of  the  new  FAA  regulations. 

2)  Hitachi's  ability  to  produce  full  daylight  in  all  natural  colors  including 
blue  brings  greater  realism,  not  only  to  the  airport  scenes  but  also  to 
enroute  flying  between  airports.  Simulator  flying  in  adverse  weather  con¬ 
ditions,  as  is  required,  takes  on  new  dimensions  of  reality  when  pilots  use 
Hitachi's  Bestview  visual  display  systems. 

Operational  Capabilities: 

The  Bestview  system  simulates  scene  information  and  is  guided  by  the  movements 
of  the  aircraft  by  a  computer  generated  image  system  and  displays  the  scenes 
on  one  to  four  channels  as  required  by  the  user. 

Landscape  and  Landing  Areas: 

In  the  Bestview  system,  with  full  bright  daylight  colors,  all  objects  can  be 
displayed  in  their  natural  colors  such  as  runways,  markings,  taxiways,  ramp 
areas,  natural  vegeration,  hills,  mountains,  blue  rivers,  blue  lakes,  blue 
oceans  and  vast  city  areas  as  well  as  realistically  portraying  small  villages 
and  towns. 

Light  Points  and  Surfaces 

The  siiiulation  of  ground  scenes  by  use  of  light  points  and  surfaces,  airport 
lighting,  city  lighting,  etc.  can  be  independently  displayed  with  4,000  light 
points  and/or  350  surfaces.  This  gives  the  greater  flexibility  in  developing 
scenes  using  both  the  light  points  and  surfaces  or  using  the  light  points  and 
surfaces  independently. 
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Weather  Conditions 


The  weather  effects,  such  as  clouds,  fog  and  lightning  can  be  realistically  dis¬ 
played  in  Bestview  scenes.  The  visibility  will  be  changed  gradually  from  a  far 
point  to  a  close  point  in  response  to  the  flight  path  being  flown. 

Simulation  Capability 

Simulation  of  terrain,  ground  obstacles  and  markings: 

Terrain,  ground  obstacles  and  markings  are  simulated  by  the  following  conditions 
due  to  the  capabilities  of  Hitachi's  advanced  computer  processing  and  hardware 
performances. 

1)  The  surfaces  are  created  by  use  of  either  3  or  4  edges. 

2)  About  450  surfaces  for  one  scene  are  in  the  data  base.  450  surfaces  are 
used  so  that  a  minimum  of  350  surfaces  can  be  displayed  to  reduce  the  over¬ 
flow  conditions  of  various  surface  distribution. 

3)  Natural  terrain,  cultivated  terrain,  buildings  -  with  lighted  windows  -  and 
other  man  made  objects  are  represented  by  using  approximately  100  surfaces 
of  the  available  450  surfaces.  Current  "Bestview"  displays  have  a  total  of 
ten  (10)  occultations  for  one  scene. 

4)  Markings  -  Bestview  simulates  markings  in  great  detail  to  aid  pilots  in 
their  approaches. 

a)  Indication  markings  -  Center  line,  end,  touchdown  zone,  edge,  and  runway 
markings  are  all  represented  in  clear  white  colors  in  contrast  to  the 
dark  colored  runways  or  black  on  light  colored  runways. 

b)  Taxiways  -  Taxiway  centerline  and  stop  positions  are  represented  in 
yellow  colors. 

Light  Points  (defined  as  light  strings) 

The  light  points  are  equally  spaced  and  have  the  same  defined  color  on  a 
straight  line.  One  independent  light  point  is  also  defined  as  a  light  string. 

There  are  15  light  point  colors  for  fixed  intensity  to  variable  intensity. 

Five  colors  are  used  for  the  polyhedrons  and  make  an  occultation  of  light 
points. 

Light  strings  can  be  selected  for  one  scene  using  600  strings  or  less.  Each 
light  string  has  255  light  spots  maximum. 

Landing  Lights  and  Taxi  Lights 

1)  The  landing  lights'  brightness  and  illumination  will  vary  according  to  the 
distance  from  the  landing  lights.  If  only  one  landing  light  is  used,  the 
illumination  is  reduced  in  half,  but  the  range  of  illumination  is  not  changed. 

2)  The  taxi  lights  illuminate  runways  and  taxiways  with  less  brightness  than 
landing  lights. 
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Scene  Mode 


The  simulation  for  scene  mode  of  day/twilight/night  is  as  follows: 

1)  Day  -  Day  is  displayed  by  the  building  up  of  surfaces  bounding  one  scene. 

1  ‘ 

2)  Night  -  Night  is  represented  by  the  light  strings  that  have  been  stored  in 
the  computer  to  make  light  spots.  Horizontal  and  seaside  lines  are  also 
displayed. 

3)  Twilight  -  Twilight  is  displayed  by  a  combination  of  surfaces  and  light 
spots. 

Field  of  View 


The  pilots  see  the  images  outside  the  cockpit  windows  through  display  units. 

The  display  units  consist  of  shadow  mask  26"  color  CRT  and  optical  components. 
Optical  components  consists  of  spherical  mirror  and  beamsplitter.  The  image 
will  be  closed  to  infinity.  The  display  channels  can  be  varied  from  one  to  four 
along  with  from  two  to  six  display  units  (windows). 

Data  Base 


50  scenes  can  be  stored  in  magnetic  disc  storage  system.  Scene  selection  is 
achieved  manually  or  automatically. 

1)  The  manual  selection  is  made  by  the  scene  select  switch  on  the  visual  control 
panel. 

2)  In  addition  to  the  take  off  and  landing  at  airports,  when  an  aircraft  flies 

a  high  altitude  route  (area),  the  following  automatic  selection  is  available: 

High  altitude  scenes  of  400  N.M.  area  are  provided.  When  the  aircraft  is 
entering  or  departing  the  area  defined  as  an  airfield  scene,  the  scene  will 
be  changed  automatically  to  correspond  with  the  flight  route.  About  one 
second  is  required  for  scene  changes.  During  the  change,  the  scene  become 
a  gray  color,  as  in  clouds. 

Instructor  Operation  Console 

The  visual  display  control  panel  controls  all  of  the  Bestview  visual  systems, 
including  adjustment  and  changes  to  the  environmental  conditions.  By  use  of  a 
color  monitor,  the  instructor  observes  the  same  scenes  as  being  viewed  by  the 
pilot  and  the  co-pilot. 

Hardware  System 

The  Bestview  visual  display  system  will  produce  computer  generated  images  and 
displays  the  scenes  on  a  raster  scan  type  26"  color  CRT  with  almost  infinity 
image  of  scenes.  The  environmental  conditions  displayed  can  be  selected  from 
the  visual  display  control  panel. 
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The  Image  Generation  Equipment:  Consists  of  Display  Processing  Equipment,  Dis¬ 
play  Image  Generation  and  Control  Equipment  to  change  the  scenes  of  Day /Twilight/ 
Night  by  using  a  computer  generated  image  and  raster  scan  system. 

The  image  generation  equipment  will  process  the  data  stored  in  the  central 
processing  unit  corresponding  to  the  position  and  posture  of  the  aircraft 
giving  this  information  on  the  display  units  for  pilot,  co-pilot  and  on  in¬ 
structor's  visual  display. 

The  display  unit  consists  of  a  shadow  mask  color  26-inch  CRT  with  collimating 
mirror  and  beamsplitter  which  provides  the  authentic  scenes  for  both  the  pilot 
and  co-pilot  with  almost  infinity  scenes. 

Display  Processing  Unit 

Display  processing  unit  consists  of  a  general  purpose  computer  and  peripherals. 
The  general  purpose  computer  provides  flight  parameters  to  the  Display  Image 
Generation  and  Control  Equipment.  The  flight  parameters  are  calculated  by  the 
aircraft's  current  position  and  attitude  and  included  the  selection  of  the 
window's  view. 

Display  Processing  Units  hardware  components  are  as  follows: 

*  CPU  SEL  32/7780  or  Perkin  Elmer  3320 

*  10  M  Byte  Moving  Head  Disc.  10/32  or  DMR  CMD  Processor  Sub-system 

*  MTU  Tape  Sub-system  800/1600  EPI  or  Perkin  Elmer  Model  456/32 

*  Display  Terminal  Perkin  Elmer  Model  550  or  CRT  Operator  Console 

Display  Image  Generation  and  Control  Equipment 

The  image  generation  and  control  equipment  will  receive  the  scene  data  (such  as 
surface  and  light  string  data) ,  parameters  to  make  a  coordinate  conversion  for 
the  aircraft  location,  data  for  color  of  each  figure,  simulation  data  for  the 
environmental  conditions,  form  the  display  processing  equipment  and  will  provide 
the  computer  generated  image  signals  corresponding  to  proper  windows. 

1)  Interface  -  Interface  will  transmit  necessary  data  to  selector,  surface 
generator,  light  spot  generator  and  color  control  equipment  from  the  display 
processing  unit  transmitting  the  following  data: 

a)  Surface  data 

b)  Light  string  data 

c)  Parameter  for  selector 

d)  Parameter  for  surface  generation 

e)  Parameter  for  light  spot  generation 

f)  Data  for  color 

g)  Data  for  selecting  the  environmental  condition 

h)  Parameters  for  various  maintenance  program. 

2)  Selector  -  Selector  has  two  kinds  of  packages  for  the  surface  (selector  1) 
and  light  strings  (selector  2),  and  receives  the  following  data  through  the 
interface: 
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a)  Parameter  to  select  the  figures  which  can  be  seen  in  each  window,  calcu¬ 
lated  by  the  display  processing  unit. 

b)  Data  for  surface  or  light  strings  defined  as  a  scene. 

Selector  will  select  the  surface  which  can  be  seen  at  each  window  in  the 
surface  data  received  by  the  parameter  as  shown  in  item  a),  and  will  trans¬ 
mit  the  surface  data  to  surface  generator. 

Light  strings:  the  selector  will  select  the  data  similarly  as  surface  data, 
and  will  transmit  the  data  to  the  light  spot  generator. 

3)  Surface  Generator 

a)  Receives  the  parameter  to  make  a  coordinate  conversion  through  the  surface 
data  interface  by  the  selector  1. 

b)  Surface  data  will  be  changed  to  the  coordinate  of  aircraft  axis  based  on 
the  eye  point  of  pilot  by  a  parameter.  The  new  coordinate  is  defined  as 
a  f ield-of-view  coordinate. 

c)  Makes  a  perspective  transformation  in  the  scene  coordinate  (defined  as  a 
scene  coordinate),  and  calculates  perspective  images  of  all  surfaces. 

d)  Converts  the  data  from  item  c)  to  the  data  to  be  displayed  in  the  raster 
scan  color  CRT,  and  provides  the  output  the  same  as  the  surface  image. 

350  surfaces  are  available. 

4)  Light  Spot  Generator 

a)  Receive  the  light  string  data  from  the  selector  2  and  parameter  to  make 
a  coordinate  conversion  from  the  interface. 

b)  Generate  individual  light  spots  from  light  string  data  and  transform  the 
scene  points  into  the  f ield-of-view  coordinate. 

c)  Make  a  perspective  transformation  of  light  points  into  the  scene  coordi¬ 
nate. 

d)  Synchronize  the  data  from  item  c)  to  raster  line  of  CRT.  4,000  light 
spots  are  generated  by  the  light  spot  generator.  Light  strings  have 
maximum  of  255  light  spots. 

5)  Color  Control  Equipment 

Colors  for  surface  images  produced  by  the  surface  generator  and  light  points 
produced  by  the  light  spot  generator  are  generated  by  color  data  that  has 
been  already  designated  on  each  figures. 

When  surfaces  or  light  spots  hide  more  than  two  other  surfaces  or  light 
spots  the  color  of  figure  having  the  higher  priority  will  be  displayed. 

The  visibility,  fog  and  landing  lights  are  displayed  with  the  control  of 
brightness  and  frequency  characteristics  by  the  data  that  has  been  selected. 


The  color  control  equipment  will  provide  natural  colors  by  combining  the  red, 
green  and  blue  colors. 

Display  Unit 

Each  display  unit  consists  of  high  brightness  26"  color  CRT  and  highly  accurate 
optical  components. 

Visual  scenes  are  displayed  on  the  CRT  receiving  the  video  signals  and  sync, 
corresponding  to  each  window  produced  by  image  generation  equipment.  The 
optical  components  used  present  the  pilots  with  an  almost  infinity  image  of  the 
scenes  displayed  on  the  CRT. 

High  Brightness  Color  CRT  -  High  brightness  CRT  display  utilizes  the  26-inch 
RGB  Shadow  Mask  CRT,  and  provides  bright  and  sharp  picture. 

High  Accurate  Optical  Component  -  Highly  accurate  optical  component  consists 
of  spherical  mirror  and  beamsplitter  installed  in  each  visual  display  unit. 

Visual  Display  Control  Panel 

Visual  display  control  panel  is  installed  in  the  instructor  console  of  the 
flight  simulator  and  controls  all  of  the  visual  display  system. 

Color  Monitor 


Monitor  is  used  for  surveillance  of  the  scenes  and  displays  the  front  and  side 
scenes  of  pilot  and  co-pilot. 

Software  System 

The  software  of  Bestview  visual  display  system  consists  of  image  generation 
program,  data  base  management  program  and  test  programs. 

Scene  Generation  Program 

The  image  generation  program  is  an  on-line  process  program  to  do  processing  in 
real  time  necessary  for  the  generation  of  visual  display  scenes.  The  program 
consists  of  the  following  sub-programs. 

1)  Interrupt  Process  Program  -  The  interrupt  process  program  inputs  the  system 
mode  of  flight  simulator  to  the  display  system  and  controls  the  program 
corresponding  to  each  system  mode,  and  also  controls  the  sequence  of  conduct 
and  computation  cycle  of  each  program. 

2)  Data  Transfer  Program 

a)  Input-output  processing  program  -  the  program  processes  the  input-output 
data  to  flight  simulator,  display  image  generation  and  control  equipment 
and  visual  display  control  panel. 

b)  Visual  display  control  panel  process  program  -  the  program  processes  the 
the  necessary  selected  conditions  such  as  environmental  conditions  of 
visual  display  control  panel. 
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3)  Scene  Control  Program 


The  program  calculated  the  changed  conditions  of  scene  due  to  location  and 
altitude  data  of  aircraft  from  flight  simulator,  and  controls  the  change  of 
scene  for  airfield  and  cloud  display. 

4)  Surface  Selection  Process  Program 

The  program  will  select  the  surface  data  and  block  of  light  strings  which 
can  be  seen  in  each  window  corresponding  to  the  location  and  azimuth  of  air¬ 
craft  relative  to  the  selected  scene  data  base. 

5)  Parameter  Process  Program  for  Coordinate  Conversion  and  Perspective  Conver¬ 
sion 

The  program  calculates  the  parameters  for  the  coordinate  conversion  from  the 
ground  to  the  f ield-of-view  and  from  the  f ield-of-view  to  the  perspective 
image  on  the  display. 

6)  Polyhedron  Process  Program 

The  program  process  to  make  the  hiding  of  surfaces  of  polyhedron  due  to  the 
data  for  position,  azimuth  and  altitude  of  aircraft  relative  to  the  map  data 
selected.  Will  also  conduct  the  occultation  process  between  each  polyhedron. 

7)  Horizon  Line  Process  Program 

The  program  calculates  the  position  of  horizon  line  for  each  window  relative 
to  the  aircraft  attitude. 

8)  Environmental  Process  Program 

The  program  processes  the  environmental  conditions  such  as  lightning,  clouds, 
visibility  and  fog  selected  by  the  visual  display  control  panel. 

9)  Subroutine  Programs 

The  programs  are  for  the  function  processing  such  as  trigonometric  function 
and  square  root  included  in  the  equations  used  for  visual  display  system 
software. 

Data  Base  Management  Program 

1)  Scene  Data  Base  Input  Process  Program 

The  program  process  on-line,  the  input  data  of  surfaces  and  light  strings  by 
using  a  card  reader,  and  loads  and  stores  them  easily  in  memory  system. 

2)  Scene  Data  Base  Management  Program 

The  program  is  used  for  the  process  of  omitting,  adding  and  re-editing  parts 
of  scene  data  base  stored  in  auxiliary  memory  system  according  to  the  re¬ 
quirements. 
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Test  Program 

Test  programs  are  used  to  test  hardware  of  the  visual  display  systems  and 
consist  of  display  processing  equipment,  display  image  generation  and  control 
equipment. 

1)  Test  Program  for  Dsiplay  Processing  Equipment 

The  program  will  test  and  check  the  operation  of  main  memory  and  digital 
components  including  peripheral  equipment  of  computer. 

2)  Test  Program  for  Display  Image  Generation  and  Control  Equipment 

The  display  processing  equipment  simulates  the  hardware  structures  of  display 
image  generation  and  control  equipment  on  software  base,  and  the  test  data  is 
given  to  the  display  image  generation  and  control  equipment  and  the  output 
data  is  fed  back  to  the  display  processing  equipment  to  compare  with  the 
result  of  software  simulation  and  to  provide  diagnostic  analysis  of  the  dis¬ 
play  image  generation  and  control  equipment. 

The  Connection  with  Flight  Simulator 


We  designed  the  connection  of  the  visual  display  system  with  flight  simulators 
to  be  very  simple.  The  major  items  in  hardware  connections  with  the  flight 
simulator  are: 


1)  Installation  of  the  display  units  in  cockpit  windows. 


2)  Installation  of  visual  display  control  panel  in  the  instructor  console. 


3)  Installation  of  image  display  equipment  in  the  computer  room. 

4)  Installation  of  interfaces  in  the  computer  room. 


The  details  of  above  items  are  determined  when  the  hardware  and  software  used 
for  the  flight  simulator  system  are  known. 

Maintenance 

The  following  items  are  provided  to  improve  the  maintenance  of  visual  display 
system: 

1)  Maintenance  -  Sufficient  space  is  available. 

2)  Tools  -  The  visual  system  has  been  designed  to  eliminate  the  use  of  special 
tools. 


3)  Self  Test  Program  -  Self  test  program  for  each  functional  block  is  provided 
to  improve  the  maintenance  of  display  system. 
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SPECIAL  FEATURES  OF  THE  BESTVIEW  VISUAL  DISPLAY  SYSTEMS 


As  image  problems  have  appeared  Hitachi  Denshi  has  developed  solutions  for 
some  of  the  problems  as  follows: 

Edge  Smoothing 

The  solution  to  the  jagginess  of  the  edges,  one  of  the  weak  points  in  raster 
scan  type  system  is  accomplished  partially  in  vertical  direction  only  in  this 
economically  priced  system  for  the  following  reasons: 

a)  In  view  of  the  effective  resolution  of  the  system  i.e.  1,024  (H)  x  483  (V) 
lines,  it  is  necessary  to  obtain  the  balance  between  the  horizontal  and  the 
vertical  resolutions. 

b)  Since  the  picture  movement  in  the  screen  is  more  often  vertical  than 
horizontal,  the  compensation  in  vertical  direction  is  more  effective  than 
horizontal. 

In  this  system,  the  smoothing  effect  is  obtained  by  producing  one  raster  having 
the  average  value  of  two  sampling  lines.  Thus,  j aggies,  stepping  and  scintil¬ 
lation  etc.  are  reduced  so  as  not  to  be  readily  apparent. 

Runway  Conditions 

Following  functions  are  provided  in  our  Bestview  visual  systems  to  simulate 
varying  runway  conditions,  such  as  patchy  wet,  patchy  icy  or  wet  on  rubber 
residue  in  the  touch  down  zone. 

1)  Indication  on  the  Display 

Colors  such  as  light  gray  or  white  are  used  to  distinguish  one  condition 
from  the  other  i.e.  dry,  wet  or  icy  etc.  If  the  condition  in  same  zone 
differs  from  the  rest  of  the  greater  surrounding  area,  it  is  indicated  in  a 
different  color  from  the  rest  by  the  data  fed  into  the  data  base  beforehand. 

2)  Output  to  the  Simulator 

A  spare  interface  having  the  capacity  of  48  words  (16  bits)  to  feed  the 
conditions  as  rubber  residue,  patchy  condition  etc.  to  the  simulator  is  also 
provided. 

Weather  Representation 

1)  Variable  Cloud  Density  and  Gradual  Breakout 

The  visibility  is  gradually  reduced  as  the  aircraft  enters  the  clouds  through 
the  cloud top  or  cloudbase  as  set  by  the  switch  on  the  control  pane.  Converse¬ 
ly,  when  the  aircraft  is  emerging  from  the  middle  of  the  clouds,  the  visibili¬ 
ty  gradually  improves  from  the  initial  state  of  clouds,  emerging  sight  of  the 
ground  and  to  the  final  visibility  level  set  by  the  visibility  switch  at  an 
altitude  below  the  cloud  base. 
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The  clouds  density  is  indicated  by  the  difference  in  the  altitude  of  aircraft 
between  the  zero  visibility  point  in  the  clouds  and  the  point  the  aircraft 
enters  the  clouds.  For  instance,  when  the  clouds  are  thick,  zero  visibility 
point  comes  just  inside  the  clouds  and  when  it  is  this,  it  is  still  visible 
at  a  point  further  inside  the  clouds. 

The  color  of  the  clouds  when  the  craft  is  totally  in  the  cloud  is  either  gray 
or  black  just  like  the  totally  invisible  state  under  day/twilight/night 
flight. 

2)  The  Effect  of  Fog  on  Airport  Lighting 

When  fog  reduces  the  visibility,  the  lights  will  be  gradually  dimmed  or 
obscured  setting  used  depending  upon  the  fog. 

3)  Category  II  and  III  Weather 

These  conditions  are  produced  by  giving  the  date  for  the  clouds  and  visibility 
as  required. 

Optional  Facility 

The  Bestview  Visual  Systems  are  provided  with  optional  capability  to  produce 
images  at  a  predetermined  position  in  the  scene. 

1)  Moving  Hazard 

The  images  moving  from  a  set  position,  consists  of  surfaces  and  light  points 
representing  a  vehicle  or  an  aircraft  which  are  produced  in  the  scene.  The 
timing  of  the  start  of  motion  is  given  from  the  instructor  operation  console 
with  a  push  button.  Work  is  now  in  progress  to  add  greater  model  realism 
with  combinations  of  CGI  and  T.V.  etc.  -  some  excellent  aircraft  air  to  air 
work  was  prepared  for  the  Japanese  military  program. 

2)  Partial  Obscuration  of  Ground  Scenes 

This  is  produced  in  the  scene  during  the  approach  and  landing  phase  of  the 
flight  below  2,000  ft  HAA  and  within  the  radius  of  10  miles  from  the  airport. 
This  is  defined  separately  from  the  clouds.  The  surfaces  of  light  gray  color 
and  the  predetermined  size  having  the  priority  over  any  other  surfaces  or 
light  points  are  placed  at  predetermined  positions  in  the  space,  and  the 
visibility  around  them  is  set  to  an  adequate  level  with  the  visibility  switch 
to  soften  the  effect.  These  clouds  are  so  positioned  so  that  no  two  clouds 
overlap. 

3)  Patchy  Fog 

During  the  final  approach  and  landing  phase  of  flights  below  2,000  ft.  HAA 
and  within  the  radius  of  10  miles  from  the  airport,  vertical  surfaces  of 
predetermined  sizes  and  having  the  priority  over  any  other  surfaces  or 
light  points  are  placed  at  predetermined  positions  on  the  ground.  The  visi¬ 
bility  around  them  is  set  to  an  adequate  level  to  soften  the  effect.  These 
are  so  positioned  so  that  no  two  fog  scenes  overlap. 
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A  PARTIAL  LIST  OF  THE  SIMULATORS  FOR  TRAINING  AND  OTHER  USES 


MANUFACTURED  BY  HITACHI  DENSHI  CO.,  LTD.,  TOKYO,  JAPAN 


1.  P-2J  Operational  Tactical  Trainer 

2.  PS-1  Weapon  System  Trainer  (Flight  Trainer) 

3.  PS-1  Weapon  System  Trainer  (Tactical  Trainer) 

4.  T-3  Flight  Simulator 

5.  C— 1  Flight  Simulator 

6.  Submarine  Tactical  Trainer  (STT) 

7.  Mine  Tactical  Trainer  (MTT) 

8.  Active  Sonar  Trainer  (AST) 

9.  Surface  Group  Anti-Submarine  Tactical  Trainer  (SATT) 

10.  General  Purpose  Flight  Simulator 

11.  Flight  Control  System 

12.  Rocket  Dynamic  Characteristic 

13.  V/STOL  Airplane  with  Visual  System 

14.  Automatic  Landing  and  Taking  Off 

15.  Space  Travel  Simulator  with  Visual  Display 
16  Dynamic  Characteristic  of  Airplane 

17 .  Warning  System  of  Airplane 

18.  N-Type  Rocket 

19.  Tracking  Tester  with  Visual  Display  System 

20.  Ship  Operation  and  Control  with  Visual  Display  System 

21.  Atomic  Power  Reactor 

22.  Strong  Earthquake  Response  Analyze  Computer 

23.  Flood  Forecasting 

24.  Cerebrum  and  Heart  Simulation  for  Medical  Use 


A  Brief  explanation  of  some  of  the  simulators  with  defense  applications: 

P-2J  &  PS-1  Operational  Weapons  System  Flight  Tactical  Trainer 

P-2J  and  PS-1  are  anti-submarine  attack  airplanes  whose  mission  is  to  search, 
detect,  discriminate  and  then  attack  enemy  submarines.  It's  training  mission 
is  to  improve  the  probability  of  target  location,  using  many  kinds  of  sensers, 
and  to  identify,  process  and  to  evaluate  the  sum  total  of  the  data  information 
received  from  the  sensers. 

P-  J  and  PS-1  Operational  Tactical  Trainers  provide  a  team  training  of  crew 
(pilot  and  five  operators)  in  the  simulator  to  improve  the  tactical  capability 
of  anti-submarine  warfare  by  P-2J  and  PS-1  airplanes. 

Crew  training,  as  a  team,  is  one  of  the  most  important  factors  of  the  simula¬ 
tor  training  programs  because  of  the  complexity  and  high  level  of  electronic 
senser  equipment  and  systems  used  for  anti-submarine  warfare. 

The  simulator  instructors  can  set  the  tactical  conditions  and  maneuvers  of 
one  or  more  targets  and  provide  the  data  for  evaluation  of  individuals  and 
teams  in  the  various  training  exercises. 
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PS-1  Operational  Flight  Trainer 


PS-1  Seaplanes  can  take  off  and  land  in  very  rough  seas.  PS-1  flight  simulator 
or  had  to  create  the  required  realism  of  rough  seas,  so  that  take-offs  and  land¬ 
ings  on  the  waves  could  be  accomplished  in  addition  to  the  ordinary  functions 
and  performances  of  a  conventional  flight  simulators. 

The  following  operational  training  programs  are  available  in  the  simulators 
just  as  they  would  be  encountered  in  the  actual  seaplane  training  exercises 
at  sea. 

a)  Taxiing  on  smooth  water 

b)  Taking  off  and  landing  on  smooth  water 

c)  Military  requirements  while  airborne 

d)  Taxiing  on  rough  seas 

e)  Taking  off  and  landing  on  rough  seas 

T-3  Flight  Simulator 

T-3  flight  simulator  provides  real  and  accurate  simulation  of  all  functions  and 
characteristics  of  the  T-3  elememtary  training  aircraft  used  for  ab  initio  pilot 
training. 

C— 1  Flight  Simulator 

The  C— 1,  2-engine  jet  flight  simulator,. with  its  Bestview  CGI/TV  full  color  and 
bright  daylight,  twilight  and  night  scenes  provides  the  simulation  for  all  normal 
functions,  performances,  and  characteristics  of  the  C-l  in  addition  to  its  many 
military  functions,  such  as,  military  equipment  drops,  paratroopers,  enemy 
radar  detection,  etc. 

The  pilots,  navigators  and  plane  captains  train  as  a  crew  to  develop  greater 
skills  of  coordination  for  the  many  highly  technical  military  operations  in  the 
simulator  costs. 

Playback  functions,  storing  operational  data  of  the  training  progress  for 
instant  freeze,  and/or  later  replays  of  the  training  exercises  will  show  the 
dynamic  variation  of  the  instruments  readings  from  the  scheduled  missions 
requirements  etc.  An  evaluation  function  stores  the  quantitative  data  for 
training  evaluation  and  types  out  the  mission's  results  upon  command. 

Submarine  Tactical  Trainer 

Hitachi  Denshi's  submarine  tactical  trainer  programs  provide  intensive  train¬ 
ing  for  all  crew  members  of  submarines  in  the  same  operations  as  on  board 
actual  submarines  on  tactical  training  missions  of  search,  detection,  class¬ 
ification,  discrimination,  tracking,  attack,  evasion,  etc.. 

Simulating  other  submarines  and  targets,  such  as  destroyers,  ships,  aircrafts, 
helicopters,  etc.  in  the  tactical  environment  required  by  the  various  tactical 
training  missions  provides  the  evaluation  of  data  of  the  training  level  of  the 
various  stations  and  crew  members  as  well  as  the  data  developed  during  the 
training  exercises. 

Mine  Tactical  Trainer 

A  major  advance  in  underwater  mine  detection  training  has  been  made  by  Hitachi 
Denshi  with  their  mine  tactical  trainer  that  simulates  all  the  tactical  train¬ 
ing  to  complete  the  various  mine  sweepers  missions. 

The  tactical  situations  are  controlled  by  the  instructors  according  to  the 
mission  requirements. 

The  evaluation  of  the  data  of  various  levels  of  the  training  exercise  is  record¬ 
ed  and  compared  with  the  actual  mission  programs. 
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SUMMARY 


The  quality  of  the  images,  the  full  range  of  colors,  the  bright  daylight,  and 
the  realism  of  moving  models  of  the  current  CGI  made  by  Japanese  companies, 
or  at  least  one  Japanese  company,  may  come  as  a  surprise  to  many  people. 

And,  according  to  several  Hitachi  Denshi  engineers  responsible  for  image 
production,  they  feel  their  present  work,  in  general,  is  as  good  or  better 
than  the  image  systems  they  have  viewed  during  their  current  visits  to  many 
airlines,  aircraft/simulator  manufactures  in  Japan,  USA,  Canada  &  Europe, 
where  they  have  looked  at  the  Vital  III,  IV,  SP-1,  SP-2,  CT-4,  DIG  and 
Compuscene  systems  being  displayed. 


CONCLUSION 


The  Japanese  have  been  very  successful  in  marketing  their  35mm  cameras,  TV  sets, 
studio  TV  cameras,  automobiles,  etc.  in  many  parts  of  the  world. 

However,  they  have  been  slow  and  patient  in  introducing  new  products,  research¬ 
ing  the  potential  markets  to  be  sure  their  products  would  meet  the  high  stand¬ 
ards  of  quality  that  people  have  come  to  expect. 

Given  time,  exposure  and  marketing  experience,  Hitachi  Denshi,  may  do  as  well 
with  their  "Bestview"  CGI  and  more  advanced  hybrid  visual  display  systems  in 
the  future. 

Thank  you  for  your  interest. 
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PANEL  DISCUSSION 


"What  You  Always  Wanted  to  Know  About  CIG 
But  Were  Afraid  to  Ask" 


Chairman 

Mr.  Herbert  A  Cooles 
Manager,  Advanced  Technology 
American  Airlines  Training  Corporation 


Mr.  Cooles  is  presently  Manager  Advanced  Technology  at  American  Airlines 
Training  Corporation.  He  has  been  actively  engaged  in  the  development  and 
application  of  simulation  technologies  for  the  past  eighteen  years.  This 
experience  has  included  almost  all  facets  of  the  science  and  art  of  flight 
simulation.  His  most  recent  activities  are  concerned  with  the  development 
of  advanced  visual  simulation  techniques  for  both  commercial  and  military 
applications. 

Mr.  Cooles  received  his  8.S.  degree  from  Texas  A  &  M  University  and 
graduate  work  at  U.S.C.  He  is  an  Associate  Fellow  of  the  AIAA. 
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PANELISTS 


Colonel  John  a  Hall 
USAF  Tactical  Air  Warfare  Center 


Colonel  John  A.  Hall  was  born  1  August  1928  in  Smithville  MS.  He 
possesses  a  Bachelor  of  Arts  from  the  University  of  Mississippi  and  a  Master 
of  Science  from  George  Washington  University. 

Colonel  Hall  was  assigned  to  USAFTAWC,  Eglin,  AFB,  FL,  on  23  August  1978 
as  the  Asst  DCS/Aircrew  Training  Devices.  He  became  the  DCS/Aircrew 
Training  Devices  on  22  January  1979. 


Colonel  Arlin  "Art”  Deel 
USA  NASA/Ames  Research  Center 


Colonel  Arlin  "ART"  Deel  is  the  Development  Project  Officer  and  the 
Army/NASA  Program  Manager  for  the  Rotorcraft  Systems  Integration  Simulator 
(RSIS),  an  advanced  R&D  simulator  to  be  used  for  rotorcraft  handling 
qualities  research  and  systems  development.  He  is  a  Master  Army  Aviator 
rated  in  both  fixed-  and  rotary-wing  aircraft  with  over  4500  flight  hours. 
He  has  completed  three  years  in  his  present  assignment. 


PANELISTS 

Commander  William  D.  Jones,  USN 
Staff,  Chief  of  Naval  Air  Training  (CNATRA) 
Corpus  Christi,  Texas 


Commander  William  D.  Jones,  USN,  has  served  in  Naval  Aviation  for  24 
years.  In  addition  to  four  tours  in  Attack  Squadrons  he  has  served  as 
training  officer  for  the  A7  Fleet  Replacement  Squadron  and  commanding 
officer  of  an  advanced  jet  training  squadron.  Currently  assigned  to  the 
staff  of  the  Chief  of  Naval  Air  Training,  he  has  been  involved  with  the 
acquisition  of  the  2F129  and  2B37  flight  simulators;  he  is  the  staff's 
program  manager  for  VTXTS  and  is  assigned  as  a  member  of  the  VTXTS  Source 
Selection  Evaluation  Board. 

Lieutenant  Colonel  Dennis  L.  Cole,  USAF 
Commander,  Detachment  3,  4200  Test  and  Evaluation  Squadron  (SAC) 
Plattsburgh  AFB,  New  York 


Lt  Col  Cole  was  commissioned  through  the  Reserve  Officer  Training  Corps 
in  1965  following  graduation  from  Rensselaer  Polytechnic  Institute,  where  he 
earned  a  Bachelor  of  Aeronautical  Engineering  Degree.  After  an  educational 
delay  to  acquire  a  Master  of  Science  in  Management  Degree  from  the  same  uni¬ 
versity,  he  entered  active  duty  in  1967.  Upon  completion  of  pilot  training, 
he  transitioned  to  the  F-100  Super  Sabre  and  flew  162  combat  missions  while 
stationed  at  Phan  Rang  AFB,  Viet  Nam. 

In  1979  he  assumed  cornand  of  Det  3,  4200  TES,  coordinating  the  unit's 
functions  of  on-site  acquisition  management  of  WST  modifications.  Develop¬ 
ment  Engineering  Prototype  Site  operations,  Software  Configuration  Manage¬ 
ment,  and  training  program  support  actions. 
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SESSION  III 
Chairman 


Lieutenant  Colonel  Bobby  R.  Adams 
Chief,  Aviation  Systems 
Naval  Training  Center 
Orlando,  Florida 


Colonel  Adams  was  born  on  May  10,  1940  in  Buford,  Georgia,  and  was  edu¬ 
cated  in  the  public  schools  in  that  city.  He  majored  in  Business  Adminis¬ 
tration  at  North  Georgia  College  and  graduated  in  1962  with  a  Bachelor  of 
Science  degree,  and  as  a  distinguished  military  graduate  of  the  Reserve 
Officers  Training  Corps  received  a  commission  as  a  second  lieutentant  in  the 
Regular  Army. 

In  June  1962,  he  entered  active  military  service  and  was  assigned  as 
platoon  leader  of  Echo  Company,  First  Airborne  Battlegroup  503d  Infantry  at 
Fort  Bragg,  North  Carolina.  From  June  1962  to  February  1964,  Colonel  Adams 
attended  the  Infantry  Officers  Basic, Course,  Airborne  and  Ranger  Schools  and 
served  as  Battle  Group  S-l.  He  attended  the  Officers  Fixed  Wing  Aviators 
Course  in  1964  and  was  assigned  to  Darmstadt,  Germany. 

Colonel  Adams  returned  to  the  United  States  in  1966  and  attended  the 
Rotary  Wing  Qualification  Course  at  Fort  Rucker,  Alabama. 

From  April  1966  to  April  1967,  Colonel  Adams  served  in  the  Republic  of 
South  Vietnam  as  a  scout  pilot.  Rifle  Company  Commander,  and  Aide-de-Camp  in 
the  First  Cavalry  Division. 

Colonel  Adams  attended  the  Project  Managers  Course  at  the  Defense  Sys¬ 
tems  Management  College  in  1979  and  assumed  his  current  duties  in  February 
1980. 

His  military  decorations  Include  the  Distinguished  Flying  Cross  with  one 
Oak  Leaf  Cluster,  Bronze  Star  with  two  Oak  Leaf  Clusters,  Meritorious  Service 
Medal,  Air  Medal  with  nine  Oak  Leaf  Clusters,  Army  Commendation  Medal  with 
two  Oak  Leaf  Clusters,  and  Vietnamese  Gallantry  Cross  with  Silver  Star. 
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TURNKEY  CAD/ CAM  SYSTEMS 
IN 

CIG  DATA  BASE  CREATION 


John  L.  "Jack"  Booker 
Acquisition  Director  and 
Principal  Investigator 
Naval  Training  Equipment  Center 
Computer  Systems  Laboratory 
Orlando,  Florida 


John  L.  Booker  is  Acquisition  Director 
and  Principal  Investigator  on  Tasks 
8741,  "Low  Level  CIG  for  Trainers"  and 
8743,  "Area  of  Interest  CIG"  for  the 
NAVTRAEQUIPCEN  Computer  Systems  Labora¬ 
tory,  Code  N-74,  Orlando,  Florida.  He 
served  as  project  engineer  for  the 
Aviation  Wide  Angle  Visual  System 
(AWAVS)  CIG  system  procurement  with 
General  Electric  Co.  which  was  deliver¬ 
ed  in  1978.  He  has  been  active  in  com¬ 
puter  graphics  since  1967  and  has 
served  as  project  engineer  on  a  number 
of  other  computer  graphics  system  pro¬ 
curements  in  the  laboratory  including 
an  IDIIOM  and  E&S  LDS-1.  He  received 
the  Master  of  Engineering  degree  from 
the  University  of  Florida  in  1967, 

BSEE  from  N.C.  State  University  in 
1961  and  AB  degree  in  Journalism  from 
the  University  of  North  Carolina  in 
1953.  Mr.  Booker  is  a  member  of  Tau 
Beta  Pi,  Eta  Kappa  Nu,  Sigma  Xi,  IEEE 
Computer  Society,  SIGGRAPH  and  the  ACM. 


Sam  T.  Giambarberee 

Programmer/Analyst 

Naval  Training  Equipment  Center 


Sam  T.  Giambarberee  obtained  his  B.S. 
in  Computer  Science  from  the  University 
of  Central  Florida  and  is  currently 
working  on  his  M. S.  degree.  His  three 
years  of  experience  in  computer  graphics 
involves  data  base  creation  techniques 
of  various  3D  models.  His  work  re¬ 
quires  use  of  the  Applicon  3D  graphics 
computer  and  the  Movie  BYU  software 
package  implemented  on  a  VAX  11/780. 
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TURNKEY  CAD/CAM  SYSTEMS  IN  CIG  DATA  BASE  CREATION 


ABSTRACT 

The  Naval  Training  Equipment  Center  experience  in  use  of  a  Turnkey  Compu¬ 
ter  Aided  Design/Computer  Aided  Manufacturing  (CAD/CAM)  System  interfaced  to 
a  widely  available  3-dimensional  image  rendering  software  package  provides 
effective  interactive  graphics  input,  creation,  modification  and  control  of 
CIG  visual  data  bases.  The  system  in  use  provides  many  interactive  features. 

It  can  produce  calligraphic  vector  drawings  and/or  continuous  tone  raster 
images  on  TV  video  tape  or  photographic  output  devices. 

INTRODUCTION 

The  Computer  Aided  Design/Computer  Aided  Manufacturing  (CAD/CAM)  segment 
of  the  computer  graphics  industry  has  seen  continuous  growth  during  the  last 
10  years.  There  are  a  large  number  of  turnkey  CAD/CAM  systems  commercially 
available  which  provide  interactive  features  for  creating  two-dimensional  (2D) 
and/or  three  dimensional  (3D)  drawings  and  data  bases.  One  characteristic  all 
of  these  systems  have  in  common  are  highly  developed  man-machine  interfaces 
which  provide  visual  feedback  tightly  coupled  to  various  interactive  I/O 
devices.  Most  of  the  turnkey  CAD/CAM  software  packages  represent  hundreds  of 
manhours  of  graphic  programming  and  human  engineering  design. 

Another  characteristic  most  of  the  turnkey  systems  share  in  common  is 
that  line  drawings  or  vector  drawings  are  supported  rather  than  continuous 
tone  polygons  and  polyhedra  usually  associated  with  computer  generated  imagery 
(CGI  or  CIG)  systems.  There  are  numerous  2D  CAD/CAM  systems  commercially 
available  which  support  applications  in  schematic  drawings,  circuit  board 
design,  and  graphic  arts.  There  are  not  so  many  which  support  3D  applications 
and  these  are  considered  top-of-the-line  systems  by  many  people. 

During  the  early  days  of  computer  graphics  development,  several  centers 
of  excellence  developed  in  the  university  community.  One  of  these,  which  was 
responsible  for  numerous  landmark  publications,  was  the  graphics  group  in  the 
Computer  Science  Department  of  the  University  of  Utah  under  Ivan  Sutherland 
and  David  Evans.  The  graphics  software  developed  at  Utah  represented  the 
latest  in  rendering  and  hidden  line  algorithm  development  and  was  a  significant 
resource  for  Utah  graduates  as  they  migrated  to  other  Universities  and  set  up 
other  graphics  groups. 

Many  of  the  good  features  of  the  graphics  software  developed  at  the  Univer¬ 
sity  of  Utah  have  been  incorporated  into  a  software  package  called  Movie  BYU 
distributed  by  Hank  Christiansen  of  the  Civil  Engineering  Department  of  Brigham 
Young  University  (BYU) .  During  the  years  when  the  graphics  software  was  being 
developed  at  the  University  of  Utah,  Dr.  Christiansen  was  serving  as  a  consul¬ 
tant  at  the  University  of  Utah  in  structural  engineering  with  the  Computer 
Science  graphics  group.  He  originally  developed  Movie  BYU  as  a  finite  element 
analysis  system  for  structural  design. 

The  Movie  BYU  software  package  is  available  from  BYU  for  a  distribution 
fee  of  $300  and  has  been  installed  on  a  number  of  different  computers.  The 
development  of  Movie  BYU  has  been  sponsored  by  a  number  of  government  agencies 
and  commercial  sponsors . 
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X 


During  the  summer  of  1978 ,  it  occurred  to  members  of  the  graphics  group  of 
the  Naval  Training  Equipment  Center  (NAVTRAEQUIPCEN)  Computer  Systems  Laboratory 
(Code  N-74)  that  the  power  and  flexibility  of  an  existing  CAD/CAM  system  in 
generating,  manipulating  and  modifying  3D  drawings  and  data  combined  with  the 
power  of  the  Movie  BYU  display  and  rendering  software  could  provide  an  excel¬ 
lent  starting  point  for  developing  an  in-house,  non-real-time  Computer  Image 
Generation  (C1G)  system  capable  of  producing  high  detailed/high  visual  quality 
images  for  training  simulation  applications.  Movie  BYU  was  Installed  on  the  in- 
house  computer  system,  a  VAX  11/780,  and  an  existing  CAD/CAM  system  used  to 
interactively  create  and  modify  3D  data.  The  body  of  this  paper,  then, 
describes  the  system  which  has  evolved  in  the  laboratory. 

BODY 


The  Naval  Training  Equipment  Center  (NAVTRAEQUIPCEN)  Computer  Systems 
Laboratory  (Code  N-74)  has  had  a  turnkey  CAD/CAM  system  in  operation  in-house 
since  the  summer  of  1978  when  an  Applicon  AGS  835  Graphics  System  was  delivered 
by  General  Electric  Company  as  part  of  the  Aviation  Wide  Angle  Visual  System 
(AWAVS)  Computer  ImageGeneration  (CIG)  System  as  part  of  the  Off-Line  Data  Base 
Creation  Facility  which  was  described  in  a  paper  presented  at  the  1977  Image 
Conference  at  Williams  AFB.^1'  This  system  has  been  used  to  develop  and  main¬ 
tain  3D  drawings  of  objects  used  in  creation  of  the  visual  environments  and 
3D  data  bases  used  by  the  real-time  CIG  system. 

The  Applicon  AGS  835  is  shown  schematically  in  Figure  1.  It  consists  of 
a  main  processor  cabinet  containing  a  minicomputer  and  display  processor,  and 
a  user  terminal  consisting  of  a  storage  scope  and  interactive  I/O  devices. 

The  main  processor  cabinet  contains  a  general  purpose  minicomputer  (DEC  PDP 
11/04)  which  maintains  the  graphic  data  base,  services  interactive  I/O  device 
interrupts,  and  generates  a  list  of  graphic  processor  display  generation  in¬ 
structions  for  the  custom  Applicon  display  processors.  The  user  terminal 
consists  of  a  local  vector  generator  and  Tektronix  storage  scope  display 
terminal,  alphanumeric  keyboard,  a  small  x,  y  digitizing  tablet,  a  large 
34x44  inch  digitizing  table  for  direct  input  from  maps  and  large  size  drawings, 
and  a  set  of  64  discrete  programmable  function  button  switches  for  graphic 
subroutining. 

The  standard  Applicon  3D  AGS  880  operating  system  is  used  to  provide 
interactive  display  control,  image  manipulation,  picture  component  additions/ 
deletions,  and  modifications.  Data  is  input  via  the  terminal  keyboard,  pro¬ 
grammable  function  buttons  and/or  the  digitizer  tablet.  Not  all  of  the  AGS 
880  features  are  required  for  our  CIG  application.  Some  of  the  capabilities 
in  use  are  the  image  viewing  transformations,  working  grid  accuracy  for  in¬ 
puts,  the  teach  mode  functions,  macro  capabilities,  and  standard  I/O  output 
format.  Some  of  the  Applicon  AGS  880  features  used  by  Code  N-74  for  manipula¬ 
tion  and  control  of  3D  images  in  CIG  data  base  development  are  described  in 
following  paragraphs. 

The  large  and  small  digitizer  tablets  provide  input  of  point  data  to  the 
graphic  data  base  by  use  of  working  grids  which  provide  accuracy  in  each  axis 
to  one  part  in  16,777,215.  The  terminal  operator  when  working  in  3D  usually 
works  in  two  orthogonal  views  either  front-top  or  front-right  side  views. 
Front-right,  for  example,  will  allow  x  and  y  to  be  specified  in  the  front  view 
or  z  and  y  in  the  right  hand  view.  A  deep  plane  for  z  can  be  specified  in  the 
right  hand  view  which  will  remain  static  until  changed  and  allows  multiple 
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points  to  be  input  from  the  front  view  with  the  same  z.  Likewise,  a  deep  plane 
in  the  front  view  provides  a  constant  x  value  for  y,  z  points  entered  in  the 
right  hand  view. 

Up  to  four  views  may  be  viewed  simultaneously,  with  front,  right,  top  and 
isometric  views  being  a  typical  four  window  format.  Isometric,  dlmetric, 
axiometric,  and  true  perspective  projections  from  any  viewing  location  may  be 
specified  and  generated  in  addition  to  standard  orthogonal  views.  Most  C1G 
applications  require  either  two  window  or  four  window  formats  of  the  primary 
orthogonal  views  and/or  isometric  views.  Drawings  may  be  edited  and  modified 
in  isometric  as  well  as  in  front,  top  and  side  views. 

An  unusual  feature  of  the  Applicon  system  is  use  of  character  recognition 
software  in  conjunction  with  the  digitizer  input  stylus.  A  teach  mode  is  pro¬ 
vided  in  which  the  operator  may  literally  teach  the  software  his  individual 
handwriting.  The  teach  mode  allows  customized  symbols  to  be  defined  for  each 
of  the  AGS  880  interactive  graphic  functions  which  are  then  entered  into  a 
custom  dictionary  for  each  operator.  The  teach  mode  also  allows  macro  defini¬ 
tion  of  sequences  of  AGS  880  commands  to  be  defined  symbolically  for  references 
by  the  keyboard  or  programmable  function  buttons  and  graphically  for  reference 
by  the  digitizer  stylus. 

One  of  the  main  NAVTRAEQUIPCEN  Code  N74  CIG  data  base  applications  has 
been  to  edit  and  modify  3D  CIG  data  bases  produced  by  GE  for  the  Visual  Tech¬ 
nology  Research  Simulator  (VTRS)  and  AWAVS  CIG  system.  The  GE  data  bases  are 
entered  into  the  Applicon  from  card  format  listings  of  vertexes,  faces  and 
object  models  using  symbolic  labels.  A  group  of  Applicon  macros  were  developed 
which  allow  the  GE  data  bases  to  be  entered  via  the  keyboard  and  yet  retain 
the  GE  label  information  in  the  internal  Applicon  data  base. 

An  Applicon  drawing  is  put  together  from  component  libraries  on  disk 
memory  of  the  computer.  The  library  is  a  user-created  collection  of  all  the 
basic  graphic  components  that  will  appear  on  the  final  drawing.  A  component 
can  be  any  graphic  entity,  simple  or  complex,  composed  of  arbitrary  3D  lines 
and  dimensions.  In  the  CIG  application  the  components  are  forced  to  be  con¬ 
structed  in  an  orderly  manner  such  that  the  end  result  is  a  closed  polygon,  or 
face,  that  can  be  manipulated  in  a  desired  fashion. 

The  Applicon  provides  15  Save  Bins  that  can  be  used  to  save  any  part  of 
a  drawing.  In  the  CIG  application,  bins  1  and  2  are  set  aside  for  construction 
of  the  drawing  (3D  data  base).  Bin  1  contains  the  current  composite  object 
being  constructed  from  multiple  faces.  Bin  2  contains  the  current  composite 
drawing  constructed  from  multiple  objects. 

When  coordinate  data  is  entered  into  the  Applicon  data  base,  heavy  use  is 
made  of  the  Applicon' s  Setpoint  facility  which  provides  graphical  calculations, 
graphical  constructs  and  allows  symbolic  assignments  to  graphical  entities. 
Vertexes  may  be  referenced  graphically  and  symbolically  as  V.,,  V2,  ....etc., 
faces  FACE1,  FACE2,  FACE3,  ....etc.,  where  each  face  is  made  up  of  V^  thru  V 
in  a  connected  sequence.  In  the  GE  data  base,  faces  are  composed  of  closed 
vectors  connected  in  clockwise  order. 

The  Applicon  macros  allow  easy  input  of  the  GE  CIG  data  base  entities, 
vertexes,  edges,  faces,  objects,  and  models.  Original  data  input  is  via  the 
terminal  keyboard  and  programmable  function  button  invocation  of  the  input 
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macros.  Graphic  image  feedback  is  provided  on  the  storage  scope  display  and 
often  shows  up  inconsistencies  in  operator  input.  The  programmable  function 
buttons  provide  flexible  control  of  the  input  sequence. 

Once  a  CIG  data  base  has  been  entered  into  the  Applicon  data  base  as  a 
drawing,  any  vertex,  edge,  face,  or  object  may  be  selected  by  the  stylus  on 
the  graphic  tablet.  Since  the  input  procedure  has  retained  the  GE  symbolic 
labels  originally  used  on  the  GE  card  input  format,  a  symbolic  designation 
for  each  vertex  along  with  x,  y,  z  coordinate  data  for  each  point  may  be  ob¬ 
tained  in  listing  format  on  the  display.  This  is  shown  in  figures  2,  3  and 
4  where  a  front  view  of  a  TA4J  is  shown  in  figure  2  with  an  edge  selected. 

The  dge  relates  to  a  face  which  is  designated  by  select  "butterflies"  in 
fife-- a  2.  One  face  containing  the  edge  is  identified  as  TACA08  for  which  a 
listing  of  the  vertexes  making  up  the  face  is  shown  in  figure  3.  Once  a 
single  face  is  symbolically  identified,  each  face  in  the  group  may  be  selected 
and  the  coordinate  data  of  its  vertexes  listed.  For  instance,  the  faces  of 
object  TACA  labeled  TACA01  thru  TACA08  may  be  individually  listed  on  the  dis¬ 
play  monitor.  Objects  such  as  object  TACA  may  be  isolated  and  displayed 
separately  as  in  figure  4. 

Once  the  object  is  selected,  it  can  be  changed  and  all  of  the  new  infor¬ 
mation  displayed.  Documentation  can  then  be  updated  to  the  new  values  pro¬ 
vided  in  the  listing  output.  Automation  of  the  update  process  is  the  next 
step  and  will  be  initiated  this  fiscal  year  in  the  laboratory. 

Applicon  drawings  may  be  output  in  hard  copy  picture  form  on  a  Tektronix 
thermal  printer  or  on  a  Calcomp  Model  1060  high  speed,  64  inch,  multi-color, 
drum  plotter.  Another  nice  feature  of  Applicon  data  bases  is  that  a  standard 
digital  I/O  format  specif ication  10  121  will  convert  an  Applicon  drawing  into 
an  ASCII  disk  jJile  which  can  be  run  over  a  communication  link  or  magnetic 
tape  to  the  Code  N-74  in-house  VAX  11/780  computer. 

Movie  BYU  was  installed  on  the  Code  N-74  VAX  11/780  during  the  summer  of 
1978.  The  software  package  was  well  documented  and  contained  a  section 
describing  interface  procedures  for  various  types  of  graphic  peripheral  de¬ 
vices  called  "Device."  The  Movie  BYU  package  was  installed  on  the  VAX/VMS 
operating  system  and  test  pictures  being  generated  on  a  Dicomed  D47  image 
recorder  within  a  month  after  receipt  of  the  Movie  BYU  magnetic  tape. 

Movie  BYU  is  a  collection  of  FORTRAN  programs  which  may  be  used  for  dis¬ 
play  and  manipulation  of  data  representing  mathematical,  architectural,  and 
topological  models  whose  geometry  may  be  described  in  terms  of  panel  (n-sided 
polygons)  and  solid  elements,  or  contour  lines.  The  program  modules  are 
entitled  DISPLAY,  UTILITY,  SECTION,  TITLE,  MOSAIC  and  UPDATE.  The  six  programs 
display  the  data  as  either  line  drawings  or  continuous-tone  images,  modify  it, 
clip  and  cap  it,  generate  title  or  geometric  model  files,  convert  contour 
definitions  into  polygonal  element  mosaics,  or  update  old  geometry  files. 

The  DISPLAY  module  is  the  heart  of  the  system  and  supplies  most  of  the 
capabilities  utilized  in  the  CIG  display  application  developed  by  Code  N-74. 
DISPLAY  is  organized  into  three  sub-modules,  COMMAND,  HIDDEN  and  DEVICE  which 
supply  interactive  commands,  hidden  line  or  hidden  surface  algorithm  options, 
and  output  display  options. 
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The  interactive  command  processor,  COMMAND,  provides  some  36  commands. 

The  number  and  type  of  output  device,  translation  and  rotation  of  both  global 
and  local  rigid  body  axis  systems,  color  and  shading  rules,  uniform,  linearly 
varying,  matching  at  boundaries,  highlights,  hidden  line,  hidden  surface  algo¬ 
rithms  ("poor  man's"  or  Watkins'),  vector  or  scaler  function  for  displacements 
or  stresses,  contour  generation,  surface  patches,  and  animation  features  are 
all  options  which  are  provided. 

The  HIDDEN  processor  provides  hidden  line  algorithm  options  for  line 
drawing  or  vector  output  formatted  display  devices  or  hidden  surface  options 
for  continuous  tone  raster  formatted  photographic  or  TV  video  output*  The 
"poor  man's"  hidden  surface  algorithm  eliminates  all  back  facing  Ivgons  and 
effectively  cuts  the  problem  size  in  half.  The  other  option  is  Mi 2  scanline 
hidden  line/surface  algorithm  developed  at  the  University  of  Utah  by  Gary 
Watkins  or  "Watkins'"  algorithm.  It  is  a  completely  rigorous  scanline  algo¬ 
rithm  which  takes  advantage  of  line-to-line  coherence  of  images. 

The  DEVICE  module  allows  selection  of  the  output  format  for  which  a  dis¬ 
play  is  to  be  generated.  The  Code  N-74  configuration  of  I/O  devices  available 
on  the  VAX  11/780  consists  of  a  Tektronix  4014  storage  scope  display  which 
accepts  vector  format  display,  and  a  Dicomed  D47  color  image  recorder  high 
resolution  raster  format  4096  lines  by  4096  pixels  per  line.  The  continuous 
tone  raster  format  allows  shading  to  be  calculated  by  one  of  three  rules.  The 
shading  may  be  uniform  over  the  element,  may  vary  linearly  over  the  element  or 
may  vary  linearly  with  shading  matched  at  the  element  boundaries  (i.e.,  smooth 
surface  simulation) . 

During  the  spring  of  1979,  the  use  of  the  Applicon  in  CIG  data  base  up¬ 
date  and  modification  mode,  and  availability  of  ASCII  files  containing  the 
Applicon  data  base  drawing  description,  coupled  with  a  working  Movie  BYU 
package  with  flexible  I/O  options  begged  for  development  of  a  translation 
program  which  would  allow  combining  the  power  of  both  systems  for  use  in 
CIG  data  base  development.  A  translator  program  which  accepted  the  Applicon 
"Apple  I/O"  ASCII  drawing  file  as  input  and  converted  it  to  the  input  format 
for  Movie  BYU  data  bases.  This  program  was  installed  and  operational  on  the 
VAX  11/780  by  early  summer  1974  and  pictures  of  a  GE  CIG  data  base  installed 
on  the  Applicon  were  being  converted  and  displayed  on  the  Dicomed  by  the 
Movie  BYU  program. 

An  interesting  capability  provided  by  the  combined  Applicon  data  base 
and  Movie  BYU  display  options  which  is  not  provided  by  the  turnkey  CAD/ CAM 
system  is  the  ability  to  display  vector  drawings  on  the  Tektronix  4014  storage 
scope  with  hidden  lines  removed.  The  Applicon  alone  cannot  do  this.  Figure 
5  shows  an  isometric  drawing  of  the  GE  CIG  data  base  of  a  TA4J  aircraft  with¬ 
out  hidden  lines  removed.  Figure  6  shows  the  same  isometric  drawing  with 
hidden  lines  removed  by  the  Movie  BYU  Watkins'  hidden  line  algorithm. 

Finally,  figure  7  shows  a  shaded  image  with  uniform  shading  over  each  surface 
generated  on  the  Dicomed  D47  camera  station. 

SUMMARY  AND  CONCLUSIONS 

The  NAVTRAEQUIPCEN  Computer  Systems  Lab  has  had  a  turnkey  CAD/CAM  system 
in  operation  since  1978  which  has  been  used  to  develop  and  maintain  3D 
drawings  (i.e.,  data  bases)  of  objects  used  in  creation  of  CIG  data  bases. 


151 


152 


This  turnkey  CAD/CAM  system  is  a  minimal  configuration  Applicon  AGS  835 
Graphics  System  which  was  supplied  as  part  of  the  AWAVS  Off-line  Data  Base 
Development  Facility  Subsystem. 

The  standard  Applicon  3D  AGS  880  operating  system  is  used  to  provide 
interactive  display  control,  image  manipulation  and  picture  component  addi¬ 
tions,  deletion  and  modifications.  Data  is  input  via  a  terminal  keyboard, 
programmable  function  buttons  and/or  digitizer  tablet.  A  group  of  Applicon 
macros  were  developed  in-house  to  aid  data  input  from  standard  CIG  data  base 
format.  The  symbols  and  mnemonics  contained  in  the  data  base  listings  are 
maintained  internally  in  the  Applicon  data  base.  These  can  then  be  inter¬ 
rogated  and  retrieved  at  any  time  on  the  Applicon  display  terminal. 

Once  a  CIG  data  base  has  been  input  as  a  drawing  on  the  Applicon  system, 
it  can  be  viewed  on  the  storage  scope  display,  manipulated,  augmented  or 
modified  using  the  interactive  features  provided  by  the  AGS  880  3D  graphic 
operating  system.  Changes  to  an  existing  drawing  may  be  listed  and  retrieved 
on  the  display  in  symbolic  form.  The  x,  y,  z  coordinates  for  any  vertex  may 
be  retrieved  with  its  associated  symbolic  tag.  Changed  coordinates  are  then 
entered  into  the  CIG  data  base  source  file  with  the  VAX/VMS  editor  software. 

Output  from  the  Applicon  may  be  obtained  in  picture  form  from  a  Tektronix 
thermal  printer  or  in  "Apple  I/O"  format  on  a  nine-track  magnetic  tape  unit. 
All  Applicon  drawings  are  in  3D  vector  format  with  no  hidden  line  removal. 
Other  display  formats  may  be  obtained  by  transferring  Apple  I/O  data  on  the 
magnetic  tape  to  the  VAX  11/780  for  further  processing. 

A  transformation  program  was  developed  to  accept  "Apple  I/O"  formatted 
data  as  input  and  produce  outputs  compatible  with  the  input  formats  required 
by  a  3D  rendering  software  package  on  the  VAX  11/780.  This  rendering  package 
is  part  of  a  larger  system  of  programs  called  Movie  BYU  which  was  installed  on 
the  Code  N-74  VAX  11/780  during  the  summer  of  1979.  The  current  rendering 
software  uses  the  January  1980  version  of  Movie  BYU. 

Movie  BYU  is  available  from  Brigham  Young  University  for  a  distribution 
fee  of  $300  and  has  been  installed  on  a  number  of  different  computers.  The 
development  of  Movie  BYU  has  been  sponsored  by  a  number  of  government  agencies 
including  ARPA,  ONR,  and  NASA  at  the  University  of  Utah,  ERDA  at  Lawrence 
Livermore  Labs  and  BYU  and  the  Corps  of  Engineers  at  BYU.  A  number  of  com¬ 
mercial  sponsors  were  responsible  for  the  most  recent  January  1980  version. 

Once  the  Applicon  data  has  been  transformed  into  Movie  BYU  input  format, 
a  large  number  of  display  options  are  available  on  the  display  equipment 
within  the  Computer  Systems  Laboratory.  Output  can  be  obtained  on  the  display 
in  vector  form  on  a  Tektronix  4014  storage  scope  display  with  hard  copy  out¬ 
put.  Hidden  line  removal  is  available  using  either  a  "poor  man's"  hidden 
surface  algorithm  which  removes  back  facing  polygons  or  the  more  accurate, 
but  computationally  Intensive,  Watkins  hidden  surface  algorithm.  Continuous 
tone  solid  object  images  may  be  obtained  on  a  Dicomed  D-47  color  image  re¬ 
corder  in  photographic  form  or  in  TV  video  format  from  an  IK0NAS  Graphics 
Systems  digital  image  frame  buffer. 

Our  experience  has  shown  that  use  of  standard  turnkey  CAD/CAM  graphics 
software  for  interactive  control  of  data  base  creation  and  modification 
processes  coupled  with  a  rendering  software  package  such  as  Movie  BYU  has 
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produced  a  powerful  computer  generated  display  facility  at  nominal  development 
cost.  The  CAD/CAM  software  provides  interactive  features  which  have  evolved 
through  more  than  a  decade  of  user  experience,  and  represents  more  than  a 
hundred  manyears  of  software  development.  Movie  BYU  represents  the  distilla¬ 
tion  of  the  best  of  the  University  of  Utah  software  developed  by  some  of  the 
"giants"  of  computer  graphics  during  their  graduate  study  years.  It  provides 
a  powerful  and  very  flexible  display  rendering  capability.  The  only  develop¬ 
ment  effort  required  in  our  laboratory  has  been  development  of  the  "Apple 
I/O"  to  Movie  BYU  format  translator  and  the  effort  to  install  Movie  BYU  on 
the  VAX  11/780  computer.  Use  of  existing  software  rather  than  custom 
development  from  scratch  represents  a  tremendous  leverage  in  the  use  of  pro¬ 
gramming  manpower.  The  power  and  flexibility  of  the  resulting  synergism 
probably  exceeds  what  our  in-house  programming  capability  could  have  produced 
over  any  reasonable  time  period. 
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ABSTRACT 


Three  visual  databases  in  the  Shuttle  Mission  Simulator  (SMS)  are  described. 
These  are  used  in  four  Digital  Image  Generation  Systems  (DIGS)  to  simulate 
the  visual  environment  of  ail  phases  of  the  space  shuttle  mission.  Solu¬ 
tions  are  described  for  various  simulation  tasks,  including  landing  sites, 
visual  landing  aids,  stars,  sun  glare  and  payloads. 


BACKGROUND 


The  mission  simulators  at  the  NASA  Johnson  Space  Center  have  been  the  pri¬ 
mary  simulators  for  flight  crew  training  in  the  NASA  manned  space  programs. 
The  Mercury  Procedures  Trainer,  Gemini  Mission  Simulator,  Command  and  Lunar 
Module  Simulators,  Skylab  Simulator,  Orbiter  Aerof light  Simulator,  and  now 
the  Shuttle  Mission  Simulator  have  all  contained  high  fidelity  representa¬ 
tions  of  spacecraft  interiors,  instruments,  and  window  visual  displays. 
Previous  visual  displays  used  starballs,  films,  and  closed  circuit  tele¬ 
vision  (CCTV )  utilizing  models  or  electronically  generated  images. 

The  demanding  payload  simulation  requirements  of  the  space  shuttle  missions 
precluded  the  use  of  models  and  required  innovative  new  techniques.  The 
SMS  contractor,  the  Singer  Company,  satisfied  these  requirements  by  a  com¬ 
pany-developed  Digital  Image  Generation  System. 


INTRODUCTION  TO  THE  SHUTTLE  MISSION  SIMULATOR  VISUAL  SYSTEM 


The  SMS  visual  system  includes  tour  DIGS.  The  Motion  Base  Crew  Station 
(MBCS)  DIGS  simultaneously  generates  three  color  real-time  scenes.  These 
three  views  are  displayed  in  any  four  adjacent  windows  of  the  six  SMS  for¬ 
ward  windows  according  to  the  SMS  operator’s  selection.  An  identical  DIGS 
supplies  the  forward  window  scenes  for  the  Fixed  Base  Crew  Station  (FBCS) 
(Figure  1).  These  two  forward  window  DIGS  use  identical  visual  databases. 
The  database  for  each  DIGS  is  stored  In  a  67  megabyte  disc  memory  unit. 

A  third  DIGS  generates  the  FBCS  black  and  white  closed  circuit  television 
scenes  from  Orbiter  payload  bay  and  Remote  Manipulator  System  (RMS)  arm 
cameras.  These  are  displayed  on  two  Orbiter  onboard  TV  monitors.  Either 
of  the  monitors  may  contain  a  split-screen  view  showing  the  centers  of 
two  camera  fields-of-views.  The  SMS  does  not  permit  simultaneous  split¬ 
screen  views  on  both  CCTV  monitors.  Thus,  up  to  three  Independent  views 
may  be  generated  by  the  CCTV  DIGS.  The  CCTV  database  is  very  different 
from  the  forward  windows  database. 
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Figure  1.  Fixed  Base  Crew  Station,  MBCS  in  background,  DIGS  in  foreground 

A  fourth  D'GS,  using  a  database  slightly  simpler  than  that  for  the  CCTV 
DIGS  generates  three  black  and  white  views  for  display  in  the  two  aft 
overhead  and  the  two  aft  payload  windows.  Two  independent  views  of  the 
payload  bay  are  used  when  attention  is  centered  there.  The  third  over¬ 
head  view  would  then  be  shared  and  displayed  identically  on  both  overhead 
windows.  But,  for  example,  when  the  mission  task  is  a  rendezvous,  the 
SMS  operator  would  select  the  overhead  windows  for  independently  gene¬ 
rated  views  with  the  aft  windows  shared. 

This  paper  will  describe  the  three  visual  simulation  databases  used  in 
the  forward  and  CCTV/aft  DIGS  rather  than  the  SMS  DIGS  hardware.  Inter¬ 
data  8/32  support  computers/perl pherals ,  special  visual  effects  genera¬ 
tion  hardware,  or  window  display  hardware.  To  give  these  database 
descriptions  full  meaning  and  to  enable  a  better  understanding  of  the 
space  shuttle  missions,  some  of  the  simulation  requirements,  techniques, 
capabilities,  problems  and  solutions  will  be  given  emphasis. 

FORWARD  WINDOWS  DATABASE  -  EARTH  ORBIT  SIMULATION 

The  earth  was  modeled  as  a  1620-sided  spherical  polyhedron  in  the  SMS  for¬ 
ward  windows  database.  Each  side  or  face  approximates  an  equilateral  tri¬ 
angle  with  edges  averaging  460  nautical  miles.  DIGS  hardware  curved  surface 
shading  prevents  visibility  of  the  triangle  edges  except  at  the  horizon. 

The  26  1:5,000,000  scale  Global  Navigation  charts  were  used  to  model  surface 
detail  with  3,000  polygons  (faces)  and  19,000  polygon  vertices  (face  bound¬ 
aries). 


Large  land  areas  were  modeled  with  fidelity  sufficient  for  recognition. 

The  level  of  detail  modeled  was  consistent  everywhere.  Thus,  smaller  is¬ 
lands  did  not  have  enough  detail  for  recognition.  Later,  some  detail 
was  added  to  enable  I ndenti f icatlon  of  candidate  contingency  landing 
site  terrain. 

Limited  success  was  achieved  in  meeting  the  goal  of  realistic  presentation 
of  vegetation  and  global  color  patterns.  The  earth's  bright  saturated 
colors  are  pleasing  to  the  eye  but  appear  somewhat  cartoon ish. 

The  earth's  horizon  provides  an  out-the-wi ndow  visual  pitch  and  roll  atti¬ 
tude  reference.  Apparent  motion  of  terrain  features  provide  a  visual  yaw 
reference.  Large  cyclonic  cloud  patterns  were  modeled  to  provide  a  visual 
yaw  reference  over  the  oceans.  Four  earth  databases  were  complied  differ¬ 
ing  in  amounts  of  cloud  cover: 

1.  Cloud-free  earth 

2.  Approximately  10?  cloud  cover  -  only  over  oceans 

3.  Approximately  25?  cloud  cover  -  almost  all  in  ocean  areas 

4.  Approximately  50?  cloud  cover  -  mostly  over  the  oceans 

Configuration  4  (Figure  2)  approximates  the  earth's  50?+  average  cloud  cover 
and  has  been  the  only  configuration  used  during  training  sessions.  For 
variety,  any  of  the  other  earth  databases  could  be  assigned. 


Figure  2.  USA  from  1800  nautical  miles  altitude.  Few  clouds  over  land. 
Extra  detail  near  White  Sands  for  Northrup  Strip  contingency  landing  site. 
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FIGURE  3.  Detailed  Edwards  TAEM/Landing  Scene  overlies  west  coast  earth 
scene. 


Edwards  AFB/Rogers  Dry  Lake  from  58,000  feet  altitude. 


RW  23,  Outer  glide  slope  triangle.  Threshold  stripe  and 
touchdown  markers  (barely  resolved). 


FIGURE  7.  Approach  to  KSC  following  a  launch  abort.  45,000  feet  altitude. 


FIGURE  8.  Launch  tower  for  the  Space  Shuttle  seen  from  the  Orblter. 
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FORWARD  WINDOWS  DATABASE  -  EDWARDS  AIR  FORCE  BASE  LANDING  SITE 


Edwards  Terminal  Area  Energy  Management  (TAEM)  and  approach/ landing  areas 
are  modeled  from  1:250,000  scale  maps  for  the  TAEM  area  to  1:4800  scale  maps 
In  the  landing  area.  This  database  lies  on  top  of  the  earth  model  and  Is 
accessed  under  150,000  feet  altitude.  The  large  TAEM  area  surrounding 
Edwards  Air  Force  Base  measures  276  statute  miles  north  to  south  and  190 
west  to  east  (Figure  3).  Experience  has  shown  this  to  be  somewhat  larger 
than  needed.  The  TAEM  includes  shoreline,  lakes,  metropolitan  areas,  major 
highways,  and  mountains  modeled  as  a  two-dimensional  flat  surface  (2D). 

Edwards  15,000  foot  concrete  runway  4-22  is  modeled,  (Figure  4)  along  with 
all  of  the  Rogers  Dry  Lake  runways  (Figure  5).  Hangers  along  the  flight  line 
were  modeled  with  the  added  dimension  of  height  (3D),  but  other  main  base 
and  residential  areas  2D.  Several  airports,  mines,  and  other  ground  fea¬ 
tures  were  modeled  to  provide  visual  cues  during  the  Orbiter’s  turn  to  final 
approach  heading.  Seven  hills  near  Edwards  Air  Force  Base  were  modeled 
3D  to  provide  a  height  cue  for  a  lakebed  runway  17  landing. 

Centerline  stripes,  touchdown  zone  marker  rectangles,  the  energy  reference 
mark  rectangle,  and  the  outer  glide  slope  aim  point  marker  triangle  are 
modeled  for  runway  23,  the  primary  orbital  flight  test  runway  (Figure  6). 
Some  of  these  features  are  also  included  on  runways  17  and  4-22.  Low  alti¬ 
tude  cues  modeled  for  runway  23  include  the  microwave  scanning  beam  landing 
system  (MSBLS)  vehicle,  a  fire  truck,  a  NASA  truck  and  tire  maks.  Run¬ 
way  4-22  has  a  1 07  face  boundary  B-52  on  the  taxi  way  and  includes  30  dis¬ 
tance  markers  for  determining  landing  rollout.  The  Edwards  TAEM/landing 
scene  was  modeled  using  11,000  face  boundaries. 

FORWARD  WINDOWS  DATABASE  -  KENNEDY  SPACE  CENTER  LANDING  SITE 


Following  delivery  of  the  SMS,  a  relatively  high-fidelity  8800  face  bound¬ 
ary  DIG  model  of  the  Kennedy  Space  Center  (KSC)  area  (Figure  7)  was  added 
by  Singer  Link  SIMCOM,  the  SMS  support  contractor  at  the  NASA  JSC. 

A  similar  modeling  philosphy  was  used.  All  KSC  landings  will  use  runway 
15-33,  constructed  for  the  space  shuttle  program.  KSC  landings  are  planned 
in  the  early  flight  test  missions  only  in  the  event  of  Return  of  Launch 
Site  (RTLS)  aborts.  In  later  operational  space  shuttle  missions  KSC 
will  be  a  primary  landing  site.  Two  structures  are  modeled  3D,  the  524 
feet  tall  Vehicle  Assembly  Building  (VAB)  and  the  launch  tower  on  KSC 
39A  (Figure  8) . 

FORWARD  WINDOWS  DATABASE  -  CONTINGENCY  LANDING  SITES 

The  space  shuttle  contingency  landing  sites  simulation  initially  used  the 
Edwards  or  KSC  scene  under  150,000  feet  altitude.  The  Edwards  or  KSC 
scene  was  rotated  In  azimuth  to  align  RW  4,  22,  or  17  as  appropriate 
with  the  heading  of  the  selected  contingency  landing  site  runway.  Later 
the  transition  altitude  to  the  Edwards  scene  was  reduced  to  45,000  feet 
to  allow  use  of  approach  cues  modeled  in  the  earth  orbit  scene.  This 
simulation  was  still  somewhat  unsatisfactory.  Flight  operations 


training  personnel  requested  that  the  runway  pattern  be  discernable 
above  45,000  feet  to  aid  approach  maneuvers.  Also,  the  switchover 
to  the  Edwards  scene,  was  somewhat  disconcerting  to  the  flight 
crewmen . 

The  solution  chosen  was  to  model  a  simple  landing  database  for  each 
of  the  four  current  contingency  landing  sites:  Northrup  Strip,  New 
Mexico;  Rota  Naval  Station,  Spain;  Kadina  Air  Base,  Okinawa;  and 
Hickam  Air  Force  Base/Honol u iu  International  Airport,  Hawaii. 

Detail  is  concentrated  in  the  runway  area.  A  minimum  of  detail  is  pro¬ 
vided  elsewhere  -  only  enough  to  establish  site  recognition,  runway 
identification,  and  approach  cues.  The  first  of  these,  Northrup  Strip 
(Figure  9),  required  only  600  face  boundaries  to  model  the  scene. 

Northrup  Strip  would  become  the  primary  landing  site  if  the  Rogers 
Dry  Lake  were  wet.  Rota  Naval  Station,  Spain,  might  be  used  follow¬ 
ing  a  launch  abort  initiated  just  prior  to  the  planned  orbital  insei — 
tion . 

FORWARD  WINDOWS  DATABASE  -  VISUAL  LANDING  AIDS 


■f 


Precision  Approach  Path  Indicator  (PAPI )  lights  were  approved  in  October 
1980  for  the  space  shuttle  program.  They  provide  a  backup  visual  in¬ 
dication  on  the  ground  of  the  outer  glide  slope  approach  flight  path 
angle.  Following  the  Orbiter's  turn  to  final  approach  heading  this 
angle  is  nominally  20  degrees  until  the  preflare  pitchup  maneuver  to 
the  1.5  degree  inner  glide  slope  is  begun  at  1750  feet  altitude. 


/ 


Figure  9.  White  Sands,  Holloman  AFB,  (Northrup  Strip  runways  unresolved) 
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PAPI ’s  are  similar  +o  Visual  Approach  Slope  Indicator  (VASI)  lights. 
Four  PAPI  lights,  spaced  25  feet  apart  are  placed  6500  feet  short 
of  the  runway  threshold  at  the  outer  glide  slope  aim  point.  These 
white  over  red  lights  give  indications  of  17,  19,  21,  and  23  degree 
glide  slope  respectively.  The  nominal  20  degree  glide  slope  ca-  be 
kept  accurate  within  one  degree  by  maintaining  the  proper  combination 
of  two  red  and  two  white  lights.  Accurate  control  of  glide  slope  is 
a  necessity  since  the  Orbiter's  landing  approach  cannot  be  aborted. 
Nine  PAP!  systems  are  scheduled:  two  for  KSC,  four  for  Northrup  Strip, 
and  three  for  Edwards. 

At  this  writing,  preliminary  work  is  being  done  to  develop  a  PAPI 
light  simulation  approach. 


SIMULATION  OF  SPACE  VEHICLES  IN  THE  FORWARD  WINDOWS  DATABASE 


The  forward  window  DIGS  is  capable  of  simultaneously  showing  any  two 
other  moving  space  vehicles.  A  simple  112  face  boundary  model  of  the 
Shuttle  Externa!  Tank  is  the  only  vehicle  modeled  thus  far  that  has 
been  used  in  training  sessions.  The  External  Tank  is  not  seen  while 
attached  to  the  Orbiter,  nor  during  a  normal  tank  release  sequence. 
However,  a  roll  maneuver  could  provide  a  side  window  view  of  the 
External  Tank,  to  verify  adequate  separation. 

Space  vehicles  that  need  to  be  seen  from  the  forward  windows  will  be 
modeled  for  the  forward  DIGS.  It  is  a  simple  operation  to  convert 
aft  database  vehicles  modeled  in  units  of  1/32  inch  to  forward  win¬ 
dow  database  using  units  of  1/32  foot.  Both  the  Inertial  Upper  Stage 
(I US)  and  the  Spinning  Solid  Upper  Stage  (SSUS)  vehicles  will  need 
to  be  seen  from  the  forward  windows.  The  Remote  Manipulator  System 
(RMS)  arms  will  not  be  simulated  in  the  forward  windows. 


STAR  SIMULATION 


Stars  are  simulated  in  the  forward  windows  and  in  the  two  overhead 
window  displays.  These  enhance  simulation  realism,  provide  attitude 
motion  cues  and  enable  backup  navigation  training  using  a  forward 
or  an  overhead  window. 

One  thousand  seventy-nine  stars  are  included  in  the  star  database 
simulating  87  constellations.  These  include  all  but  twelve  stars 
listed  in  the  American  Ephemeris  and  Nautical  Almanac,  which  range 
between  -  1.6  to  +  4.7  magnitude.  Thirteen  dimmer  stars  down  to 
5.6  magnitude  were  added  to  the  constella^  ons  of  Ursa  Minor,  Grus, 
Corona  Australis,  and  Crater  to  make  them  more  identifiable. 

The  SMS  stars  look  very  realistic  and  have  received  excellent  crew 
acceptance.  The  constellations  can  easily  be  identified  even  though 
they  fall  short  of  the  full  stellar  brightness  range.  The  forward 
window  stars  are  projected  using  only  the  green  guns  of  the  color 
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CRT's.  The  stars  do  not  appear  green,  however,  because  of  their 
small  size  and  low  brightness. 

A  recurring  problem  in  the  forward  window  displays  has  been  the 
variance  of  star  brightness  from  one  day  to  another.  Often  the 
dimmer  stars  cannot  be  seen.  A  workaround  solution  was  to  create 
three  new  star  databases.  In  the  first  the  dimmer  stars  are 
boosted  one-half  magnitude  in  brightness.  In  the  second  database 
the  boost  is  one  magnitude;  in  the  third,  two  magnitudes.  The 
brightest  stars  are  kept  at  maximum  TV  brightness.  Thus,  in  each 
of  the  three  new  databases  the  brightness  range  is  compressed  making 
the  dimmer  stars  more  visible.  The  appropriate  database  may  be  se¬ 
lected  prior  to  each  day's  training  session,  or  changed  during  a 
session,  if  needed. 

The  Hughes  Light  Valve  projectors/Farrand  pancake  window  displays 
made  stars  difficult  to  see  in  the  overhead  windows.  The  solution 
chosen  was  to  use  2x2  picture  element-sized  stars.  Even  though  the 
brighter  stars  look  like  marbles,  constellations  can  still  be  readily 
identified.  Stars  are  not  simulated  in  the  CCTV  s/item  or  in  the 
payload  bay  windows  where  payload  bay  lighting  would  wash  out  any 
stars.  Also,  no  need  exists  to  simulate  planets. 


SUN  GLARE  SIMULATION 


Solar  glare  :s  important  in  space  shuttle  missions.  The  first  space 
shuttle  orbital  flight  test  is  scheduled  to  launch  into  an  early  morn¬ 
ing  sun  which  would  prominently  shine  into  the  spacecraft  commander's 
left  quarter  window. 

Simulation  of  the  sun  and  moon  as  one-half  degree  diameter  discs  proved 
unsatisfactory.  Lunar  maria  and  craTer  rays  were  Initally  modeled  in 
the  front  window  DIGS  to  distinguish  the  moon  from  the  sun.  These 
lunar  features  were  too  small  to  be  unambiguously  discerned.  The  sun 
remained  virtually  indistinguishable  from  the  moon.  Worst  of  all,  its 
small  dim  television  image  appeared  pitifully  insignificant  buried  in 
the  blackness  of  space  (like  the  moon  appears  in  Figure  10).  In  con¬ 
trast,  the  photo  image  of  the  sun  in  Gemini  10  photo  S— 66— 46 111  (Fig¬ 
ure  11)  appears  as  a  10  degree  diameter  ball  of  light.  Effects  of 
sunlight  are  seen  on  the  window  15  to  20  degrees  from  the  sun. 

The  solution  implemented  was  to  provide  a  large  sun  glare  model  (Figure 
12).  A  yellowish  1/2  degree  diameter  "real  sun"  (omitted  in  the  CCTV- 
Aft  black  and  white  systems)  is  buried  in  a  10  degree  diameter  white 
solar  glare  circle.  This  is  enclosed  by  a  13  degree  diameter  circle. 

The  largest  circle,  35  degrees  in  diameter,  simulates  sun  glare  effects 
17.5  degrees  from  the  solar  center.  Realism  is  further  enhanced  because 
the  sun  glare  model  usually  washes  out  the  stars  elsewhere  in  the  window. 
For  atmospheric  flight  the  dimmer  two  rings  are  eliminated. 
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Figure  12.  Space  Shuttle  Orbiter  with  sun  glare  model  in  background. 

The  sun  glare  model  is  also  valuable  to  warn  space  shuttle  trainees 
and  SMS  instructors  when  a  CCTV  camera  is  being  pointed  near  the  sun. 
Pointing  at  the  sun  in  flight  would  probably  damage  a  camera  sun  sensor. 
This  sun  glare  simulation  has  received  good  flight  crew  acceptance. 


DATABASES  FOR  THE  CCTV  AND  AFT  WINDOWS  DIGS 

The  CCTV  and  aft  window  databases  are  similar.  They  differ  greatly 
from  the  forward  windows  database.  The  detailed  earth  of  the  forward 
windows  is  simulated  merely  as  a  featureless  white  circle.  A  darker 
circle  is  accessed  for  the  earth's  nightside. 

The  CCTV  and  aft  window  databases  focus  on  meeting  payload  operations 
requirements.  The  Orbiter  exterior  and  payload  bay  are  modeled. 

Operation  of  the  payload  bay  doors  and  radiator  panels  are  simulated. 

One  or  two  RMS  arms  can  be  simulated.  Three  payload  configurations 
have  been  simulated  (Figures  13,  14,  and  15),  and  include: 

Config.  1.  Development  Flight  Instrumentation  (DFI)  containers  and 

pallet.  Induced  Environmental  Contaminators  Monitor  (IECM) 

nni } q,  2.  DFI -IECM  (800  face  boundaries) 

One  RMS  arm  mounted  on  the  port  side.  The  RMS  will  deploy 
♦he  IECM  to  measure  thruster  plume  contamination. 

■‘♦ire  o*  Science  and  Terrestrial  Applications  (OSTA-1 ) 

...  "pr*  pa1  let  (1200  face  boundaries). 


Figure  15.  RMS,  moving  to  grapple  IECM  behind  PFTA.  Earth  in  background. 


Config.  3.  DFI/IECM,  RMS  port  arm  (600  face  boundaries) 

Payload  Flight  Test  Article  (PFTA)  (1500  face  boundaries) 

Deployment,  retraction,  and  jettison  of  the  RMS  arm(s)  is  simulated. 

The  RMS  arm  is  modeled  using  seven  independently-moving  components: 
the  Manipulator  Positioning  and  Retention  Mechanism  (MPRM) ,  the  shoul¬ 
der,  the  upper  arm,  lower  arm,  wrist  pitch  segment,  wrist  yaw  segment, 
and  the  end  effector.  The  RMS  wrist  and  elbow  cameras  are  simulated 
including  the  elbow  camera  pan/tilt  unit  motion. 

Two  forward  and  two  aft  payload  bay  bulkhead  CCTV  cameras  are  simulated 
including  motion  of  their  pan/tilt  units.  Separate  SMS  hardware  is  used 
for  special  camera  effects  and  generation  of  alpha-numerics  displayed 
on  the  CCTV  monitors. 

Simple  models  of  the  payload  bay,  RMS,  and  payloads  are  used  in  a  hard¬ 
ware/software  solution  for  solving  real-time  occlusion  priorities.  These 
occlusion  models  can  be  displayed  for  debug  purposes  instead  of  the  nor¬ 
mally  visible  mode I s . 
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DIFFERENCES  BETWEEN  THE  CCTV  AND  AFT  DATABASES 


It  Is  physically  possible  for  the  RMS  arm  to  see  the  nose  of  the  Or¬ 
ta  I  tor  or  even  underneath,  so  these  features  are  modeled  for  the  CCTV 
database.  This  detail  cannot  be  seen  from  the  aft  or  overhead  win¬ 
dows  so  it  can  be  eliminated. 

It  was  discovered  that  unseen  detail  modeled  in  the  Orbiter's  nose 
and  tail  sections  place  an  additional  load  on  the  DIGS  hardware. 
Thus,  detailed  payload  scenes  could  cause  scan  line  overflows  in 
the  CCTV  DIGS,  which  is  limited  to  256  Intersections  per  scan  line. 
Orbiter  exterior  and  payload  bay  database  simplifications  are 
planned  to  prevent  such  overflows. 


DATABASE  MODIFICATIONS 


New  databases  are  generated  using  a  digitizing  tablet/menu  system 
similar  to  other  computer  generated  image  systems.  Databases  can  also 
be  altered  in  real-time  using  an  on-line  color  modification  (OLCM)  pro¬ 
gram.  Parameters  that  can  be  changed  with  the  OLCM  program  include 
color,  intensity,  resolvability  code  and  switching  distance.  Use  of 
the  OLCM  program  reduces  database  debug  time  by  enabling  rapid  obser¬ 
vation  of  database  parameter  changes. 

Effort  is  underway  for  an  IUS  mission  deploying  the  Transmission  and 
Data  Relay  Satellite  (TORS)  and  for  a  demanding  SSUS  mission.  The  SSUS 
mission  includes  three  SSUS  vehicles  with  tilt  mechanisms,  spin  tables, 
and  sun  shields.  The  SSUS  mission  requires  twelve  individually-moving 
components  in  the  payload  bay,  three  of  which  may  move  simultaneously. 

Future  effort  will  continue  to  be  simulation  of  new  space  shuttle  pay- 
loads  to  meet  specific  mission  training  requirements. 


CONCLUDING  REMARKS 


The  development  of  computer  generated  image  technology  to  its  present 
state  has  been  rewarding.  Computer  generated  imagery  has  proved 
indispensi ble  in  satisfying  visual  simulation  requirements  for  the 
space  shuttle  missions.  The  SMS  visual  databases  provide  greater 
simulation  capability  than  previous  simulators;  ard  they  have 
the  capacity  to  be  easily  expanded  to  meet  future  requirements. 
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NAP  OF  THE  EARTH  (NOE)  MANEUVERING 
WITH  COMPUTER  GENERATED  IMAGERY  (CGI) 


INTRODUCTION 

It  has  proven  to  be  almost  impossible  to  design  theoretically  a 
digital  data  base  for  a  specific  training  task  for  Nap  of  the  Earth 
(NOE)  maneuvering.  The  eye  is  very  particular  about  the  data  it 
processes  and  will  use  cues  without  consciously  being  aware  of  what 
data  is  being  obtained  from  the  scene.  Many  unusual  perceptual 
effects  become  evident  only  when  the  scene  is  viewed  on  the  display. 
It  is  through  these  observations  that  changes  and  improvements  to 
the  digital  visual  system  are  made.  Some  examples  of  these  unfore¬ 
seen  perceptual  effects  are  "floating"  trees  that  required  a  shadow 
under  them  to  fix  them  to  the  ground,  rectangular  object  faces 
rather  than  triangular  faces  to  give  pilots  the  correct  perceptual 
cues  (1),  rotor  blades  that  "strobed"  until  modeled  by  switching 
45°  blade  patterns  each  TV  frame,  and  the  addition  of  checkerboards 
to  terrain  faces  to  indicate  changes  of  slope  as  we  will  demonstrate 
in  the  movie  that  follows.  This  paper  serves  to  illustrate  that 
the  methodology  employed  to  provide  necessary  flying  cues  was 
laborious  and  exhaustive,  yet  even  in  retrospect  it  appears  there 
is  no  better  method  presently  available. 

Initial  pilot  reactions  to  helicopter  NOE  visual  scenes  were  dis¬ 
couraging.  Although  the  scenes  contained  what  had  originally  been 
considered  sufficient  detail  the  pilots  found  it  difficult  to 
perform  many  of  their  requisite  tasks.  Low  altitude  maneuvering 
(4'  to  50'  skid  height)  was  particularly  substandard  in  that  slight 
changes  in  terrain  slope  were  extremely  difficult  to  detect. 

It  is  obvious  that  the  maneuvering  areas  should  be  enhanced  to 
provide  more  landing  information  for  the  pilot.  However,  simply 
adding  scene  detail  to  make  the  model  more  complex  may  not  contri¬ 
bute  any  additional  landing  cues  to  the  pilot.  In  fact,  the 
addition  of  this  detail  may  overload  the  computing  capability  of 
the  image  generator.  It  is  necessary,  therefore,  to  determine 
precisely  what  task-relevant  information  will  increase  the  perfor¬ 
mance  capabilities  of  the  pilot. 

The  straightforward  solution  wa^  to  provide  additional  cues  in  the 
near  field  to  aid  the  observer.  When  the  decisions  had  to  be  made 
on  the  selection  and/or  elimination  of  objects  it  was  realized  that 
numerous  compromises  were  hecessary  and  there  was  almost  no  existing 
information  to  aid  in  making  these  decisions.  Of  paramount  importance 
was  determining  the  contribution  that  2D  and  3D  objects  made  to  low 
altitude  maneuvering,  the  optimum  spacing  of  these  objects,  and  the 
relevance  of  the  type  of  2D  or  3D  objects  (trees,  logs,  stumps, 
checkerboards,  etc.).  Geometric  patterns,  either  checkerboards  or 
light  points,  can  be  generated  relatively  easily  and  usually  minimize 
CGI  loading.  Hence,  an  effort  would  be  made  to  incorporate  these 
elements  if  they  proved  useful. 
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EQUIPMENT  USED 

The  problems  of  visual  data  base  evaluation  were  compounded  by  the 
unavailability  of  the  entire  helicopter  simulator  for  the  long 
periods  of  time  necessary  for  the  lengthy  scene  debugging  and 
subsequent  tests.  The  decision  was  made  to  utilize  the  R&D  CGI 
unit  located  at  the  LINK  facility  in  Sunnyvale,  California.  This 
research  CGI  unit  has  a  single  infinite  image  display  window  and 
uses  the  same  color  CRT  as  production  units.  It  can  display  4000 
edges  with  256  intersections  per  scanline  at  256  levels  of 
occultation.  More  recent  CGI  improvements  (anti-scintillation, 
detail  faces,  visibility,  fading)  have  not  been  incorporated  ; 
however  these  factors  were  not  the  subject  of  this  evaluation. 

The  research  visual  system  has  a  control  panel  which  was  programmed 
to  simulate  a  simple  helicopter.  This  permits  dynamic  evaluation 
of  the  scene  during  actual  maneuvering  as  opposed  to  remote  static 
observation.  The  control  panel  was  also  used  to  modify  object 
switching  distances  and  to  add  or  delete  particular  objects  from 
the  scene  during  maneuvering. 

TEST  PROCEDURE 

A  low  relief  section  of  terrain  was  selected  from  a  production  data 
base  and  digitized  for  use  in  the  laboratory  unit.  This  particular 
area  was  chosen  because  of  its  subtle  peaks  and  valleys.  The  shade 
of  green  in  this  scene  was  that  most  prominently  used  in  production 
data  bases.  The  area  digitized  was  approximately  6,000'  X  4,000'. 

The  maximum  height  of  the  terrain  is  450'. 

Each  pilot  was  permitted  sufficient  time  to  become  thoroughly 
acquainted  with  the  performance  of  his  "aircraft"  before  his 
position  was  initialized  to  a  point  just  off  the  edge  of  the  terrain. 
He  then  was  asked  to  "fly"  at  varying  heights  above  the  terrain 
over  a  loosely  prescribed  course.  Estimated  heights  were  contin¬ 
uously  compared  to  a  computer  readout  of  height  above  terrain. 
(Forward  speed  and  several  other  factors  were  also  available  but 
were  not  used  in  these  particular  tests.) 

Before  each  run  (and  in  some  cases  during  a  run)  the  visual  scene 
was  modified  in  color,  type  and/or  size  of  objects,  and  density  of 
objects.  In  most  cases  this  involved  the  substitution  of  an  entirely 
new  data  base.  Pilot  performance  was  compared  for  all  combinations 
of  data  base. 

The  actual  eye  height  readings  and  the  analysis  of  the  height 
deviations  in  each  case  have  not  been  made  a  part  of  this  report 
because  the  tests  were  run  in  several  different  time  periods  and 
with  different  subjects.  Instead  we  produced  a  short  movie  of  most 
of  the  experiemental  data  bases  to  indicate  the  extent  of  these 
tests  and  to  show  subjectively  how  the  data  base  choices  for  a  NOE 
visual  scene  must  be  made. 
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MOVIE  PREPARATION 


The  movie  recording  of  each  of  these  tests  was  taken  in  real-time 
but  in  stop  action.  That  is,  all  of  the  flight  parameters  for  each 
TV  frame  were  recorded  on  disc  during  a  real-time  flight  over  the 
data  base.  In  movie  mode  the  position  data  for  each  frame  is  read 
back,  the  scene  generated  in  real-time,  the  movie  camera  shutter 
opened  for  approximately  one  second,  the  shutter  closed,  and  the 
positions  for  the  next  TV  frame  read  in.  It  takes  almost  two 
seconds  to  record  each  frame  or  about  sixty  times  longer  to  record 
the  movie  than  to  replay  it.  Several  of  the  data  bases  evaluated 
involving  real-time  moving  arrays  projected  onto  the  terrain  could 
not  be  movie  recorded  because  the  array  and  movie  mode  could  not  be 
calculated  simultaneously  in  the  research  CGI  control  computer. 

DATA  BASES/SCENES  EVALUATED 

Various  experiments  were  performed  on  the  low  relief  terrain 
copied  and  modeled  from  a  production  data  base.  These  experiments 
resulted  in  a  composite  data  base  which  seemed  to  contribute  most 
significantly  to  NOE  maneuvering. 

1.  Random  Intensity  and  Color  Alterations:  One  of  the  first 

experiements  was  to  randomly  alter  the  intensity  of  each 
terrain  face  to  break  up  the  large  homogeneous  green  areas. 

Some  interesting  visual  effects  were  noted  in  that  when 
maneuvering  at  low  altitudes  some  of  the  hills  would  appear  as 
valleys  and  vice-versa.  Some  of  these  same  effects  have  been 
noted  in  stereo  viewers  and  other  photogrammetric  equipment. 
These  effects  were  not  predictable  and  the  film  recording  did 
not  necessarily  convey  the  same  visual  effect.  Randomly 
altering  the  color  of  the  terrain  faces  was  also  discarded 
because  it  destroyed  the  terrain  cues  obtained  by  the  normal 
intensity  computation. 

2.  Accentuating  Sun  Vector:  In  the  original  data  base  the  break 
between  terrain  faces  was  almost  indiscernible  when  small 
slopes  were  modeled.  The  pilot,  unable  to  detect  a  significant 
color  change,  could  not  detect  peaks  and  valleys.  To  accentuate 
the  face  intensity  differences,  the  computation  was  altered  to 
make  the  intensity  a  function  of  the  square  of  the  cosine  of 
the  angle  between  the  face  normal  and  the  sun  vector.  The 
slope  change  has  not  been  significantly  accentuated  in  those 
slight  slope  change  areas. 

To  further  accentuate  the  intensity  differences  the  cube  of  the 
cosine  of  the  angle  between  sun  and  face  normal  was  used.  In  the 
movie  it  can  again  be  observed  that  in  the  area  of  little  slope 
change,  the  face  intensity  differences  did  not  change  enough  to  aid 
peak  and  valley  detection.  In  the  area  of  large  slope  changes,  the 
face  intensity  differences  became  so  pronounced  that  the  terrain 
became  almost  black  and  white.  This  approach  did  not  seem  to  have 
sufficient  merit  for  improving  NOE  maneuvering. 


3. 


Cross-stitching;  In  tlie  continuing  effort  to  accentuate 
terrain  slope  changes  for  the  pilot,  several  data  bases  were 
designed  with  narrow  lines  modeled  transverse  to  the  scene 
edges.  These  lines  were  quite  effective  for  this  purpose  as 
can  be  seen  in  the  movie.  However,  the  lines  were  somewhat 
disconcerting  to  the  pilot  and  expensive  in  terms  of  using 
available  scene  detail.  (Light  and  dark  contrasting  lines 
were  used  in  this  experiement  as  well  as  in  all  others.  The 
moderately  dark  lines  were  universally  preferred). 

4.  Edge  Outlining;  In  another  experiment  in  accentuating  slope, 
line  features  were  placed  at  the  boundaries  of  all  terrain 
faces.  As  before,  both  light  and  dark  contrasting  lines  were 
used.  These  lines  proved  quite  effective  in  delineating  slope 
changes  but  did  not  provide  any  cues  when  the  pilot  was  at  a 
low  altitude  within  the  terrain  face.  Since  they  were  line 
features  (two  pixels  wide  at  all  ranges)  they  appeared  overly 
dark  and  large  at  all  reasonable  ranges  and  faded  when 
approached  at  very  close  range.  Narrow  faces  rather  than  line 
features  become  larger  as  they  approach  the  eyepoint  but  were 
objectionable  because  of  the  number  of  scene  edges  required 
and  because  of  image  breakup  at  longer  ranges. 

5.  Point  Feature  Outlining;  To  make  the  scene  more  aesthetically 
pleasing  while  maintaining  the  advantage  of  line  feature 
outlining,  point  lights  were  substituted  for  line  features.  As 
with  line  features,  there  was  no  noticeable  improvement  of  the 
scene.  The  system  expense  in  terms  of  edges  required  actually 
increased. 

6.  Face  Patterns  Light  Points;  Neither  point  nor  line  outlining 
of  the  terrain  faces  aided  maneuvering  when  the  eyepoint  was 
within  the  borders  of  large  terrain  faces  ( 600-800 ' /edge ) .  To 
provide  cues  within  these  areas  light  points  were  randomly 
distributed  over  the  faces  (as  always  both  light  and  dark 
contrast  was  evaluated).  Many  average  spacings  were  modeled; 
100'  between  light  points  is  the  spacing  recorded  on  the  film 
strip.  Spacing  greater  than  100'  put  the  light  points  too  far 
apart  to  provide  any  cueing.  All  practical  spacings  were  much 
too  expensive  in  terms  of  system  capacity. 

Patterned  light  points  were  also  evaluated  (some  in  the  form  of  a 
small  cross  in  the  center  of  the  large  faces).  These  provided 
additional  position  cues  because  of  convergence  cues  but  did  not 
provide  enough  position  data  to  warrant  their  inclusion  in  the  data 
base.  (Line  features  in  the  center  of  the  larger  faces  were  also 
evaluated  with  similar  results.) 

The  constant  pixel  size  (1  x  1  or  2  x  2)  of  point  and  line  features 
causes  them  to  fade  in  the  near  field  almost  to  the  point  of  being 
indiscernible . 

7.  Face  Patterns  -  Checkerboards;  There  has  always  been  some 
reluctance  to  use  geometric  patterns  on  terrain  because  they 
would  appear  "unnatural".  These  patterns  are,  however,  the 
most  practical  to  implement  and  lend  themselves  to  automated 
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data  base  entry.  Many  combinations  of  checkerboard  sizes  and 
colors  were  modeled  and  evaluated.  The  largest  checkerboard 
(500')  was  of  little  value  since  it  was  only  slightly  smaller 
than  the  average  face  size.  Since  the  checkerboard  size  could 
be  varied  easily,  all  sizes  down  to  50'  were  modeled  and 
displayed  (500',  200'  and  100'  were  filmed  for  this  presentation). 
VJith  the  50'  checkerboard  the  terrain  scene  became  so  "busy" 
that  it  disconcerted  the  operator.  At  high  yaw  rates  with  the 
small  squares  one  could  easily  become  nauseous.  Repeated 
flights  over  the  checkerboards  narrowed  the  optimum  square 
size  to  between  100'  and  200'.  130'  was  used  in  all  further 

experiments . 

The  checkerboard  appeared  to  provide  sufficient  distance  and  near 
field  cues  to  aid  maneuver  close  to  the  ground  without  flying 
through  the  terrain.  However,  they  did  not  provide  any  cue  as  to 
the  absolute  height  above  the  ground.  Without  previously  being 
taught  the  absolute  size  of  the  checkerboards  their  size  could  have 
been  estimated  from  1000'  to  10'  depending  on  the  size  of  the  hills 
themselves. 

8.  Checkerboard  with  Stumps:  Previous  programs  evaluating  NOE 
objects  had  narrowed  the  choice  of  acceptable  objects  to 
trees,  stumps,  and  posts.  (Other  objects  such  as  rocks, 
shrubs,  bushes,  and  groundcover  were  singularly  unsuccessful). 

The  checkered  data  base,  was  populated  by  stumps  (dimensions 
approximately  2'  x  2'  x  3')  at  a  density  of  1  per  20,000  sq. 
ft.  The  stumps  served  the  purpose  of  establishing  an  absolute 
size  of  each  square  but  caused  the  pilots  to  search  for  and 
fly  from  stump  to  stump. 

9.  Checkerboard  with  Trees:  Thirty-foot  high  trees  (without 

trunks)  were  substituted  for  stumps  to  determine  if  the  choice 
of  objects  were  significant.  The  trees  were  easier  to  find 
and  made  the  terrain  more  realistic,  but  they  did  not  provide 
enough  parallel  lines  at  their  tops  and  bottoms  to  give  adequate 
height  cues.  As  with  stumps,  the  pilot  tended  to  fly  from 
tree  to  tree . 

10.  Composite  Data  Base  (Checkerboard,  trees,  stumps,  posts):  A 
composite  scene  utilized  those  data  base  features  which  seemed 
to  contribute  most  significantly  to  NOE  maneuvering.  In  the 
composite  data  base: 

1.  The  checkerboard  gives  cues  of  terrain  shape. 

2.  The  trees  are  relatively  large  objects  which  provide  a 
scale  for  the  checkerboard. 

3.  The  stumps  established  the  ground  surface. 

4.  The  posts  give  precise  height  cues  by  size  and  obser¬ 
vation  of  their  parallel  top  surfaces. 

11.  Rotor  Shadow:  The  final  composite  was  useful  for  maneuvering 
but  sparse  areas  remained  in  the  data  base.  Ideally  the  data 
base  should  be  evenly  and  densely  populated  with  objects, 
however,  the  number  of  edges  necessary  for  this  purpose  would 
far  exceed  the  computing  capacity  of  any  CGI  system.  In  an 
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attempt  to  provide  near  field  cues  at  all  times  a  translucent 
rotor  shadow  was  projected  onto  the  terrain  immediately  below 
the  aircraft  (flat  terrain  was  used  for  the  initial  experiments). 
As  the  eyepoint  approached  the  ground  the  shadow  would  appear 
in  the  window.  The  pilot  soon  learned  that  he  could  use  the 
image  of  the  shadow  in  his  window  to  maintain  relative  position. 
It  was  later  determined  that  he  was  not  actually  obtaining 
visual  cues  from  the  ground  but  simply  using  the  shadow  images 
as  a  position  indicator. 

A  number  of  other  "shadow"  patterns  with  different  colors  and  sizes 
were  evaluated  on  flat  ground  planes.  All  provided  an  excellent 
mechanism  for  maneuvering  but  lacked  realism  and  authenticity. 

12.  Ground  Pattern;  It  was  evident  that  some  kind  of  ground 

pattern  immediately  about  the  eyepoint  could  provide  the  cues 
necessary  for  extremely  close  maneuvering.  An  attempt  was 
made  to  replace  the  moving  object  rotor  shadow  with  a  moving 
object  radial  pattern  of  light  points.  This  pattern  proved  to 
be  a  little  disconcerting  at  high  yaw  rates  and  several  other 
patterns  were  modeled.  The  projection  of  the  patterns  onto 
the  terrain  slopes  was  also  implemented.  Treating  the  pattern 
as  a  moving  object  made  the  projection  computation  quite 
reasonable  for  a  single  slope  but  when  many  slopes  intersected 
at  a  point  the  number  of  patterns  involved  made  the  computation 
difficult. 

Our  efforts  to  implement  the  computation  became  so  lengthy  that  it 
was  determined  that  an  array  of  points  or  faces  could  be  computed 
in  real-time  and  ground  mapped  onto  the  terrain  easier  than  treating 
the  patterns  as  moving  objects. 

A  computer  program  to  generate  and  project  checkerboards  of  5,  10 
and  20-foot  spacing  and  two-element  light  points  of  5,  10  and 
20-foot  spacings  was  written.  Twenty  elements  were  used  in  all 
directions  from  the  eyepoint  giving  an  array  size  of  200,  400  and 
800  feet.  These  patterns  were  used  in  flights  over  the  original 
terrain  with  all  other  combinations  of  the  terrain  and  objects 
discussed  previously. 

All  pilots  were  impressed  with  the  additional  near  field  detail  and 
their  ability  to  maneuver  confidently.  The  terrain  could  be  flown 
with  only  the  projected  array  or  with  any  of  the  features  previously 
tested.  There  was  disagreement  on  the  "best"  of  the  patterns. 

Some  pilots  felt  that  the  small  computed  checkerboards  were  "busy" 
and  tended  to  "swim".  This  was  pc  rticularly  true  of  the  5-foot 
checkerboard.  Discussions  resolvec1  the  "best"  pattern  to  be  the 
10-foot  spacing,  two-element  light  joints  for  the  following  reasons. 

First,  the  light  points  do  not  obstruct  the  terrain;  secondly,  its 
extent  was  large  enough  that  it  did  not  give  the  same  type  of  cue 
as  the  shadow  pattern.  Third,  the  light  points  provide  a  more 
subtle  pattern  that  a  pilot  can  accept  as  simulated  rocks  or  gravel. 
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The  work  done  above  illustrates  the  methodology  required  to  produce 
task-related  data  bases.  It  should  be  noted  that  usually  it  was 
not  possible  to  predict  the  outcome  of  the  experiments  in  terms  of 
visual  perception. 

It  is  clear  to  all  of  us  designing  data  bases  that  it  will  be  a 
slow,  thoughtful  and  laborious  procedure  to  produce  future  digital 
data  bases.  Much  has  been  learned  by  ourselves  and  by  others 
working  on  the  same  task.  One  day  this  data  and  the  wealth  of 
observations  may  be  codified,  analyzed,  and  a  methodology  developed, 
but  until  then  we  continue  to  build,  observe  and  build  again. 
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INCREASED  SE»’30R  SIMULATION  CAPABILITY  AS  A  RESULT  OP  IMPROVEMENTS 
TO  THE  DIGITAL  LANDMASS  SYSTEM  (DLMS)  DATA  BASE 


ABSTRACT 


The  Defense  Mapping  Agency  is  producing  digital  culture  and  terrain 
elevation  data  bases  to  support  advanced  aircraft  simulators.  Data  base 
analyses  and  sensor  simulations  have  led  to  improvements  in  the  specifi¬ 
cations  for  these  data  bases,  and  have  been  used  to  define  parameters  to 
be  included  in  a  prototype  data  base  designed  to  support  visual  and  other 
high  resolution  sensors.  These  improvements  and  new  parameters  are 
discussed,  and  sensor  simulations  and  data  base  displays  are  shown. 

INTRODUCTION 


In  order  to  support  advanced  aircraft  simulators  by  providing  an  improved 
low  level  radar  training  capability  offered  by  digitally  generated  radar 
landmass  images,  the  Defense  Mapping  Agency  (DMA)  is  producing  digital 
culture  and  terrain  elevation  data  bases.  The  DMA  has  conducted  extensive 
investigations  in  digital  sensor  simulation  for  the  purpose  of  establishing 
an  effective  editing  and  analysis  capability  of  these  data  bases  (Faintich, 
1979,  1980) .  The  DMA  is  also  pursuing  advanced  techniques  for  data  base 
production  through  its  research  and  development  program.  As  a  result  of 
the  technology  developed  for  aircraft  radar  simulator  support,  multi-sensor 
scenes  and  data  bases  are  being  developed  for  both  simulators  and  input 
to  reference  guidance  systems.  The  primary  purpose  of  this  paper  is  to 
describe  results  obtained  from  improvements  to  the  Digital  Landmass  System 
(DLMS)  Data  Base  and  compare  these  results  with  both  actual  and  previously 
simulated  sensor  imagery.  This  paper  does  not  necessarily  represent  DMA 
policy,  nor  is  it  a  commitment  to  produce  data  in  a  particular  manner. 

CURRENT AND  ANTICIPATED  DATA  BASE  CONTENT 


The  current  (July  1977  specification)  DMA  standard  production  data  bases 
(Level  I)  contain  large  area  cultural  information,  and  digital  terrain  data 
sampled  at  a  three  second  interval.  The  cultural  data  consists  of  point, 
linear,  and  areal  features  described  by  characteristics  such  as  surface 
material  category,  generic  identification,  predominant  height,  structure 
density,  and  percentages  of  roof  and  tree  cover.  The  cultural  data  is 
in  lineal  (planimetric  boundary)  format  and  although  feature  sizes  may  vary 
depending  upon  local  circumstances,  the  data  reflects  a  resolution  on  the 
order  of  500  feet.  Smaller  features  are  aggregated  into  homogeneous 
features  described  by  predominant  characteristics.  The  current  high 
resolution  (Level  II)  data  bases  contain  small  area  cultural  information, 
and  digital  terrain  elevation  data  sampled  at  a  one  second  interval. 

This  translates  to  a  resolution  of  about  100  feet,  with  smaller  features 
also  aggregated.  Detailed  information  is  available  in  Reference  2, 

"Product  Specifications  for  Digital  Landmass  System  (DLMS)  Data  Base". 


182 


The  terrain  elevation  data  is  produced  by  contour  digitization  from  charts 
or  directly  from  stereo  pairs  of  photographs  using  advanced  analytical 
stereoplotters.  The  cultural  data  is  produced  from  both  charts  and 
photographs  with  a  much  higher  level  of  manual  effort  required  in  order 
to  perform  the  complex  feature  analysis.  Because  of  the  labor  intensive 
nature  of  the  task,  the  production  of  Level  II  cultural  data  ranges  from 
10  to  100  times  the  production  cost  of  Level  I  data,  depending  on  the  area. 
The  current  Level  I  data  base  program  covers  roughly  20  million  square 
nautical  miles,  with  estimated  data  base  completion  dates  in  the  1985  to 
1990  time  period.  Level  II  data  is  programmed  only  for  small  selected 
areas  of  interest. 

The  DLMS  data  bases  have  been  shown  to  be  adequate  for  support  of  long  and 
medium  range  radar  simulation  (see  Figure  1),  and  for  short  range  radar 
simulation  where  Level  II  data  is  available.  In  addition,  these  data  bases 
have  shown  some  applicability  for  multi-sensor  simulation.  As  integrated 
chip  and  microcomputer  technology  improves,  however,  on-board  multi-spectral 
electro-optical  navigation  sensors  continue  to  acquire  and  display  terrain 
and  cultural  information  with  ever  increasing  resolution  approaching  that 
of  a  visual  response.  Training  is  required  in  the  use  of  advanced  aircraft 
displays,  including  forward  looking  infrared  systems,  low  light  level 
television,  ultra-high  resolution  and  real-time  synthetic  aperture  radar, 
as  well  as  an  increased  demand  for  visual  training  associated  with  low 
altitude  mission  profiles.  Technology  improvements  will  also  allow  greater 
realism  in  advanced  simulators,  and  the  demonstrated  effectiveness  of 
present  simulators  is  driving  requirements  for  simulators  to  support  the 
new  generation  of  navigation  sensors. 

In  addition,  the  commitment  of  the  DOD  to  the  development  of  various  types 
of  weapon  system  guidance  correlators  is  demanding  new  support  data. 
Correlation  may  be  made  against  pre-stored  computer  generated  reference 
scenes  for  optical,  infrared,  microwave,  conventional  and  synthetic 
aperture  radar,  and  ultra-violet  electro-optical  sensors. 

A  crucial  component  in  support  of  these  new  navigation/guidance  systems  is 
the  ability  to  provide  adequate  digital  data  bases  that  describe  a  variety 
of  global  ground  truth  conditions.  Both  static  and  time-varying  information 
such  as  texture,  thermal  and  near-infrared  properties,  precise  geometric 
properties,  road  patterns,  population  and  traffic  density  patterns,  and 
atmospheric  weather  data  will  be  required  in  addition  to  current  DLMS 
feature  descriptors.  Along  with  new  types  of  feature  descriptors,  increased 
resolution  and  level  of  detail  will  be  required. 

Initially,  much  of  the  advanced  data  descriptors  will  not  be  cost  effective 
or  possible  to  collect  and  will  have  to  be  modeled  from  known  parameters. 

As  automated  feature  analysis  and  extraction  techniques  become  developed, 
an  increasing  amount  of  these  data  types  will  be  described  more  closely  in 
agreement  with  their  physical  characteristics  and  will  be  included  in  the 
cultural  data  base.  As  an  interim  procedure,  or  for  data  base  areas  where 
higher  resolution  data  is  required  only  for  increased  data  content  and 
appearance,  and  not  for  reasons  of  precise  ground  truth,  computer  techniques 
such  as  texturization  (Bunker,  1978)  and  synthetic  feature  break-up 
(Faintich,  1979)  allow  for  the  production  of  large  data  base  coverage  using 
existing  techniques. 
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(d)  Actuol 


(c)  Simulated 

Figure  1 .  Laval  I  DIMS  Radar  Simulations:  Las  Vegas,  Nevada 

(a)  Long  Range  (85  n.m.)  Simulation  versus  (b)  Actual  Radar; 

(c)  Medium  Range  (32  n.m.)  Simulation  versus  (d)  Actual  Radar 
Scope  Photography 


'VOLUTION  OF  DATA  BASE  PRODUCTION 


The  labor  intensiveness  of  the  present  digital  data  base  production  process 
and  the  worldwide  extent  of  DLMS  requirements  have  led  DMA  to  explore  auto¬ 
mation  as  a  means  of  decreasing  production  costs.  This  is  being  accomplished 
within  the  technical  base  existing  at  DMA  and  via  research  and  development 
programs  through  academic,  industrial,  and  governmental  institutions. 

Initial  goals  to  increase  production  of  current  DLMS  Level  I  and  II  data  are 
being  addressed  by  the  implementation  of  specialized  automated  processing 
systems  and  computer  assisted  photo-interpretation  stations.  In  order  to 
establish  the  capability  to  produce  anticipated  data  types  and  resolutions 
required  by  1985,  the  DMA  is  expanding  its  Image  Understanding  program  for 
the  technology  base  required  to  support  the  development  of  an  interactive, 
semi-automated  feature  analysis  production  system. 

For  purposes  of  quality  control  and  data  base  applicability  investigations, 
the  DMA  is  developing  the  Sensor  Image  Simulator,  a  very  high  speed  data  base 
edit  station  and  static  scene  simulator  that  allows  for  interactive  query  of 
individual  features  in  the  simulated  sensor  scene  to  determine  the  corres¬ 
ponding  data  base  elements  responsible  for  the  simulated  features. 

In  advance  of  the  integration  of  future  automated  feature  analysis  techniques, 
the  DMA  has  refined  the  DLMS  data  base  specifications  commensurate  with 
current  production  capabilities  to  better  support  sensor  simulation.  The 
1974  data  base  specifications  were  revised  in  July  1977,  and  the  DMA  is 
currently  producing  a  prototype  data  base  to  support  high  resolution  sensors. 

1977  DLMS  DATA  BASE  SPECIFICATION  REVISION 

As  a  result  of  an  intense  test  and  evaluation  of  the  data  produced  for  USAF 
Project  1183  (Robinson  et,  al,  1979),  the  DLMS  data  base  specifications  were 
changed  to  include  the  following  major  revisions: 

a.  Standardization  of  feature  descriptors  where  actual  differences  are 
insignificant; 

b.  Decrease  in  the  minimum  size  for  strong  reflectors; 

c.  Decrease  in  the  minimum  size  for  building  groupings  with  significant 

height  differences; 

d.  Increase  in  the  minimum  size  for  poor  reflectors; 

e.  Decrease  in  the  minimum  size  for  open  areas  with  an  urban  area; 

f.  Expansion  and  clarification  of  unique  feature  specifications; 

g.  Reduction  from  six  levels  (IA,  IB,  IIA,  IIB,  IIIA,  IIIB)  to 
two  (I,  II); 

h.  Standardization  of  percent  roof  cover  descriptor; 

i.  Portrayal  of  dense  trees  in  urban  areas; 
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j.  Increase  of  feature  identification  codes  from  48  to  255; 

k.  Portrayal  of  certain  regional  features  apparent  on  low  altitude  radar; 

l.  Description  of  seasonal  and  natural  effects  such  as  sea  states; 

m.  Portrayal  of  permanent  snow  and  ice. 

For  comparison  between  the  data  bases,  DLMS  cultural  and  terrain  data  over 
Spokane,  Washington,  was  chosen.  This  particular  area  was  chosen  because  of 
existence  of  both  1974  and  1977  specification  Level  II  data  over  a  large 
area,  availability  of  actual  short  range  radarscope  photography,  and  exclusion 
of  this  data  in  the  test  and  evaluation  study  that  resulted  in  the  1977 
revisions.  The  comparisons  between  Levels  I  and  II  data  may  be  somewhat 
biased,  however,  as  the  Level  I  data  was  produced  using  the  Level  II  as  a 
base.  Although  this  is  a  standard  technique.  Level  II  data  over  a  large 
area  is  usually  unavailable  to  be  used  as  a  base,  and  the  Level  I  data  is 
compiled  directly  from  feature  analysis  source  materials.  Figure  2  shows 
on-line  110  meter  resolution  data  base  displays  of  this  area,  wherein  shaded 
relief  of  Level  I  terrain  is  shown,  and  gray  levels  represent  radar  reflec¬ 
tivity  potentials.  The  synthetic  break-up  (SBU)  data  bases  were  produced  by 
generation  of  random  cultural,  tree  cover,  and  background  features  with 
descriptors  normally  distributed  about  original  predominant  values  within 
homogeneous  areal  feature  boundaries,  based  upon  the  percentages  of  tree 
and  roof  cover. 

Inspection  of  these  displays  clearly  shows  the  additional  features  portrayed 
due  to  decreased  standards  for  strong  and  significant  reflectors  (Level  I) 
and  the  reduction  of  features  portrayed  due  to  increased  standards  for  poor 
reflectors  (Level  II).  Also,  the  reflectivity  potential  for  general  resi- 
dential/comnercial  areas  is  reduced,  probably  due  to  standardization  of 
height  descriptors.  The  SBU  displays  show  the  breakup  of  the  areal  features 
into  background,  trees,  and  brighter  cultural  reflectors  due  to  elimination 
of  adjustment  for  tree  cover  within  a  homogeneous  area.  Also  of  particular 
interest  is  that  the  differences  between  the  1977  Levels  I  and  II  displays 
at  this  resolution  (for  >  15  n.m.  radar  scope  displays)  are  much  less  than 
in  the  1974  displays.  This  raises  questions  as  to  the  need  of  Level  II  data 
at  this  resolution  for  short  range  simulation.  For  short/short  range  displays 
(<  10  n.m.),  the  differences  are  much  more  apparent. 

Now  consider  Figures  3  and  4.  Radar  simulations  from  the  six  data  bases  are 
compared  with  actual  radar  scope  photography  on  15  and  22  n.m.  range  displays. 
The  gain  settings  for  all  six  of  the  simulations  in  each  figure  are  identical 
(except  for  Figures  4f  and  4g)  and  were  chosen  for  the  best  match  between 
actual  and  the  1974  data  base  simulations .  Note  that  the  1977  data  bases 
provide  for  a  better  comparison  with  actual,  especially  better  for  Level  I. 

In  addition,  for  both  Levels  I  and  II,  the  general  residential/commercial 
homogeneous  returns  are  either  missing  under  insufficient  gain  or  appear  as 
large  homogeneous  "blobs"  without  the  benefit  of  SBU.  The  SBU  simulations 
shown  in  Figures  4f  and  4g  have  slightly  lower  gain  settings  to  better  match 
actual  returns. 
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(a)  1974  Level  I 


(b)  1974  Level 


(c)  1977  Level  I 


(d)  1977  Uvil 


(•)  1977  Level  I  SBU 


(f)  1977  Level  II  SBU 


Figure  2.  On-Line  Data  Bose  Displays:  Spokane,  Washington 


(c)  1977  Lavol  I  (d)  Actual  (a)  1977  Laval  II 


(ff)  1977  Laval  I  SBU  (g)  1977  Laval  II  SBU 

Flgura  4.  Short  Ranga  (15  n.m.)  Radar:  Simulations  Using  Various 
Data  Basas  varsus  Actual  Radar  Scopa  Photography 


Figures  5  and  6  show  data  base  displays  and  radar  simulations,  respectively, 
for  the  1977  data  bases  with  and  without  SBU  at  a  resolution  of  25  meters  for 
5  n.m.  short/ short  range  displays.  Note  that  the  differences  between 
Levels  I  and  II  are  quite  apparent,  and  that  the  SBU  technique  continues  to 
add  to  the  effect  of  realistic  simulation  of  the  large  areal  features. 

One  may  conclude  frcm  these  studies  that  unless  short/short  range  simulation 
is  required  for  high  precision  conventional  radar  mission  training.  Level  I 
DLMS  data  may  be  sufficient,  and  that  the  technique  of  synthetic  break-up  adds 
to  the  realism  of  the  simulation.  For  areas  surrounding  high  fidelity 
training  data  bases,  synthetic  break-up  may  be  used  to  generate  compatible 
frequency  data  for  blending  purposes. 

When  considering  non-conventional  radars  such  as  Synthetic  Aperture  Radar 
(SAR) ,  the  high  precision  of  the  Level  II  data  may  be  required  for  much 
longer  ranges.  Figure  7  shows  SAR  simulations  from  the  same  25  meter 
resolution  data  bases.  Note  that  the  effect  of  synthetic  break-up  adds 
greatly  to  the  realism  in  the  simulations. 

PROTOTYPE  HIGH  RESOLUTION  DATA  BASE 


The  1977  revision  of  the  DLMS  data  base  specifications  includes  descriptors 
that  may  not  be  needed  for  radar  systems,  but  are  certainly  of  value  for 
simulation  of  other  sensors.  Figure  8a  is  a  visual  simulation  of  Port 
Angeles,  Washington,  and  snow-capped  Mt.  Olympus  from  a  10,000  foot  altitude, 
and  Figures  8b  and  8c  compare  shipboard  visual  simulation  with  actual 
photography.  Figure  9  compares  a  black  and  white  copy  of  a  high  altitude 
NASA  photograph  using  color  infrared  (IR)  film  over  Washington,  D.C.  Since 
a  suitable  IR  model/data  base  has  not  yet  been  developed  within  DMA,  this 
image  is  compared  to  a  visual  reflectance  data  base  over  the  same  area. 

Note  the  good  agreement  between  the  features.  Note  also  the  absence  of 
major  roads  such  as  the  Capitol  Beltway  about  Washington. 

In  order  to  support  high  resolution  sensor  simulation,  the  DMA  is  producing 
a  prototype  data  base  (reference  3)  for  various  sensors  including  visual, 

SAR,  Low  Light  Level  TV,  and  IR.  This  data  base  production  is  for  evaluation 
of  content  requirements  and  production  cost  analyses.  Large  area  (Level  V) 
characteristics  are  currently  specified,  but  small  area  (Level  X)  specifi¬ 
cations  for  high  detail  portrayal  are  not.  In  general.  Level  V  supplements 
Level  I  DLMS  with  additional  feature  descriptors,  additional  feature  identi¬ 
fication  codes  such  as  major  transportation  lines,  and  divides  concrete/ 
asphalt  into  separate  surface  material  categories.  The  new  feature 
descriptors  are  the  roof  descriptor,  shape  code,  and  microdescriptor. 

The  roof  descriptor  portrays  ten  roof  types  and  four  monitor  types,  yielding 
40  different  roofs.  The  shape  codes  are  rectangular  parallelepiped, 

(hemi-) spherical,  pyramid,  cone,  cylinder,  and  other.  The  microdescriptor 
is  a  multi-purpose  descriptor  which  describes  some  of  the  visual  character¬ 
istics  of  a  feature.  This  may  be  used  to  support  synthetic  break-up  in  a 
manner  more  realistic  than  by  purely  random  means  based  on  statistical 
percentages.  The  microdescriptors  include  vertical  composition  information 
such  as  height  and  location  of  a  tower  on  a  building,  homogeneous  area 
information  such  as  the  predominant  dimensions  of  smaller  features  aggregated 
into  larger  areal  features,  and  pattern  definition  where  applicable  such 
as  road/street  patterns. 
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(c)  1977  Laval  I  SBU  (d)  1977  Laval  II  SBU 

FI gun  5.  On -Lin*  Doto  Bos*  Displays:  Spokana,  Washington 
High  Resolution  Grid  (25  malar) 
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(c)  1977  Level  I  SBU 


(d)  1977  Level  II  SBU 


Figure  6.  Short /Short  Range  (5  n.m.)  Radar  Simulations 


>)  DIMS  Lml  N  SAR  Simulation  (b)  DIMS  Laval  II  SBU  SAR  Simulation 

DIMS  Synthetic  Aporturo  Radar  Simulations:  Spokano,  Washington 


(a)  Visual  Simulation  from  10,000  foot  Altitudo 


(b)  Visual  Simulation  from  Shipboard  Laval 


(c)  Actual  Photograph  from  Shipboard  Laval 


Figura  8.  Port  Angeles,  Washington  and  Mt.  Olympus:  Comparison 
Between  Visual  Simulations  from  DLMS  Laval  I  Data  Bases 
and  Actual  Photography 
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(a)  High  Altitude  Infra-red  Photograph 


(b)  DIMS  Level  I  Visual  Reflectances 

Figure  9.  DIMS  Data  versus  Infra-red  Photography: 
Washington,  D.C. 
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After  completion  of  the  prototype  Level  V  data  bases,  nev  sensor  simulation 
investigations  will  begin.  Previous  investigations  (Faintich,  1979) 
demonstrated  the  power  of  synthetic  break-up  in  visual  simulations  using 
statistical  percentages,  and  the  new  Level  V  data  should  provide  for  a  much 
greater  sense  of  realism  and  fidelity. 

CONCLUSIONS 


The  DMA  is  continually  striving  within  production  constraints  to  improve 
the  accuracy  and  level  of  detail  in  the  DLMS  data  bases.  Automated  feature 
analysis  techniques  are  being  developed  to  aid  in  the  collection  of  the  data, 
and  improved  sensor  simulations  technology  is  helping  to  better  utilize  the 
information  contained  in  the  current  DLMS  data  bases. 

REFERENCES 


1.  Bunker,  W.  M.;  1978;  "Training  Effectiveness  Versus  Simulation  Realism", 
Eleventh  NTEC/Industry  Conference  Proceedings,  Orlando,  Florida, 

14-16  November  1978,  pp  291-298. 

2.  Defense  Mapping  Agency;  1977;  "Product  Specifications  for  Digital 
Landmass  System  (DLMS)  Data  Base",  St.  Louis  AFS,  Missouri. 

3.  Defense  Mapping  Agency;  1979;  "Product  Specifications  for  a  Prototype 
Data  Base  to  Support  High  Resolution  Sensor  Simulation",  Washington,  D.C. 

4.  Faintich,  M.  B.;  1979;  "Digital  Sensor  Simulation  at  the  Defense  Mapping 
Agency  Aerospace  Center",  Proceedings  of  the  National  Aerospace  and 
Electronics  Conference  (NAECON) ,  Dayton,  Ohio,  15-17  May  1979;  pp  1242-1246. 
(Reprinted  in  SPIE  Proceedings,  Volume  238,  paper  238-49,  1980.) 

5.  Faintich,  M.  B. ;  1980;  "Digital  Sensor  Simulation:  1980-1985", 
Proceedings  of  the  13th  Annual  Simulation  Symposium,  Tampa,  Florida; 

19-21  March  1980;  pp  181-190. 

6.  Robinson,  R.  et.  al;  1979;  "Test  and  Evaluation  of  USAF  Project  1183 
Digital  Data  Base",  DMAAC  TR  79-1,  St.  Louis  AFS,  Missouri. 


196 


SESSION  IV 


Chairman 

John  8.  Sinacori 
President,  Sinacori  Associates 
Hollister,  California 


Mr.  John  B.  Sinacori  is  an  experienced  aeronautical  technologist  operat¬ 
ing  as  both  an  engineering  consultant  and  contracted  principal  investigator. 
His  experience  in  Aerospace  firms  includes  flight  dynamic  and  aerodynamic 
analyses,  design  of  aircraft,  rotorcraft  and  spacecraft,  research  on  pilot- 
vehicle  dynamics  and  all  aspects  of  flight  simulation  including  their  con¬ 
duct  for  training  and  vehicle  development,  simulator  design  and  simulator 
research. 

As  the  head  of  his  own  firm,  he  has  been  active  in  the  design  and  de¬ 
velopment  of  advanced  simulation  equipment,  research  on  simulator  cueing  and 
particularly  the  determination  of  motion  and  visual  cueing  needs  for  train¬ 
ing  and  engineering  development  simulation  and  the  design  and  development  of 
advanced  astronomical  telescopes. 

He  holds  a  B.S.  in  Aeronautical  Engineering  from  Rensselaer  Polytechnic 
Institute  and  has  performed  numerous  additional  studies  in  electronics, 
optics,  physics,  mathematics,  engineering  and  is  a  rated  commercial  pilot. 


PROJECT  2363: 

LIQUID  CRYSTAL  LIGHT  VALVE  PROJECTOR 


Peter  C.  Baron  obtained  his  M.  S.  in  electrical 
engineering  from  Purdue  University.  His 
26  years  of  experience  at  Hughes  have  included 
circuit  engineering,  advanced  display  develop¬ 
ment,  air-defense  system,  display  project 
engineering,  and  management  of  a  variety  of 
system  engineering  hardware  development, 
study  and  proposal  efforts.  For  the  past 
11  years  he  has  headed  the  Hughes  Ground  Sys¬ 
tems  Group  (Fullerton)  display  IR&D  program 
covering  hardware,  device  and  software  projects. 
Since  1976  he  has  had  technical  responsibility  for 
a  series  of  AFHRL  programs  (the  latest  being 
Project  2363)  to  apply  the  liquid  crystal  light 
valve  technology  to  in-line  infinity  optics  type 
simulator  visuals. 


Uzi  Efron  received  his  Ph.  D.  in  Solid  State 
Physics  from  Tel- Aviv  University  in  1976.  His 
B.  S.  and  M.  S.  degrees  are  also  from  Tel-Aviv 
University.  Dr.  Efron  is  currently  the  acting 
head  of  the  Image  Processing  Devices  Section  of 
the  Hughes  Research  Laboratories.  He  is 
responsible  for  the  development  of  both  the  photo- 
activated  and  the  electronically-addressed  silicon 
light  valves,  as  well  as  for  the  applications  of 
these  devices  in  image  processing  and  display 
systems.  From  1974  to  1977,  Dr.  Efron  was 
with  the  Department  of  Non-Destructive 
Testing  and  Radiography  at  the  Soreq  Research 
Center,  in  which  he  served  as  Head  of  several 
projects  dealing  with  Radiographic  Image 
Processing. 


Jan  Grinberg  received  his  Ph.  D.  in  Physics  from 
the  Weizmann  Institute  of  Science  in  1970.  He 
also  has  a  B.S.  and  an  M.  S.  in  Electronics  from 
the  Technion-Israel  Institute  of  Technology. 

Dr.  Grinberg  is  currently  the  Acting  Manager  of 
the  Exploratory  Studies  Department  of  HRL.  The 
department  conducts  research  in  the  areas  of 
image  analysis  software  and  hardware,  intelligent 
systems,  image  and  signal  processing  devices, 
liquid  crystal  materials,  e-beam  and  ion-beam 
resists,  holographic  gratings,  and  space  batter¬ 
ies.  He  was  previously  responsible  for  exploring 
and  developing  novel  liquid  crystal  light  valve 
configurations  using  CdS  and  silicon  photo¬ 
conductors,  and  novel  3-D  microelectronics 
structures  for  processing  of  image  information. 


I 


I 


i 


PROJECT  2363:  LIQUID  CRYSTAL  LIGHT  VALVE 
PROJECTOR  INVESTIGATIONS 

PETER  C.  BARON,  DR.  UZI  EFRON  AND  DR.  JAN  GRINBERG 
Hughes  Aircraft  Company 

ABSTRACT 

The  requirements  of  Project  2363  —  for  a  full  color  noninterlace  refreshed  display  with 
very  high  resolution  for  the  background  and  the  multiple  targets  -  make  extreme  demands 
on  the  visual  system.  Results  of  a  system  definition  study  resulting  in  a  compliant 
3"  light-valve-based  liquid  crystal  light  valve  projector  are  briefly  described.  Accom¬ 
plishments  of  the  subsequent  Component  Development  Phase  (CDP)  of  the  program, 
which  was  to  develop  the  three  high-risk  components  (very  high  resolution  3"  fiber  optic 
CRT,  3"  polarizing  prism,  and  3"  cadmium  sulfide  light  valve)  needed  to  implement  this 
project,  are  summarized.  The  construction,  operation  and  current  performance  capabil¬ 
ity  of  the  silicon  light  valve  —  the  key  to  fast  response  simulation  applications  -  is  then 
described,  and  the  results  of  projector-level  resolution  tests  conducted  on  the  Project  2363 
CDP  breadboard  projector  are  discussed.  In  summary,  the  project  successfully  demon¬ 
strated  the  feasibility  —  using  interim  cadmium  sulfide  and  silicon  photoconductor 
light  valve  configurations  -  of  achieving  the  resolution  needed  to  build  a  2'  of  arc 
projector  (at  20%  MTF). 

INTRODUCTION 


The  objective  of  Project  2363  was  to  make  a  major  advance  in  the  state  of  the  art  of  sim¬ 
ulation  visual  systems.  Specifically,  it  was  to  develop  the  computer  image  generation  (CIG) 
and  visual  system  technology  to  provide  the  capability  to  present  a  full  color,  60  hertz 
noninterlaced  refreshed  display  using  an  In-Line  Infinity  Optical  System  (ILIOS)  of  the 
general  type  used  on  the  Advanced  Simulator  for  Pilot  Training  (ASPT)  at  Williams  Air 
Force  Base,  Phoenix,  Arizona.  The  system  was  intended  to  feature  a  background  raster 
having  a  minimum  required  resolution  of  4'  of  arc  (at  the  pilot’s  eye  point)  and  a  design 
goal  resolution  of  3'  of  arc ,  and  was  planned  to  have  a  capability  of  displaying  three  small 
field  of  view  (less  than  3°  x  3°)  target  rasters  (located  anywhere  within  the  pilot's  field  of 
view)  having  a  minimum  required  resolution  of  2'  of  arc  and  a  design  goal  resolution  of  1' 
of  arc.  Figure  1  illustrates  the  requirements  for  this  Bystem.  The  visual  system  was 
to  provide  a  minimum  modulation  of  20%  at  these  resolutions  and  present  an  image 
having  a  brightness  of  6  fL  or  greater  as  viewed  through  the  ILIOS. 

It  should  be  noted  that  these  requirements  went  far  beyond  the  then  current  state  of  dis¬ 
plays  art.  A  window  of  a  dodecahedron  ILIOS  cockpit  represents  a  90°  field  of  view  (if  one 
provides  for  overscan  to  allow  for  6  inches  of  pilot  head -movement).  A  4'  of  arc  back¬ 
ground  raster  then  corresponds  to  1330  scan  lines,  to  be  painted  at  a  60  hz  rate  —  a 
significant  challenge.  A  2'_of  arc  target  resolution  corresponds  to  approximately  2700  TV 
line  resolution  across  the  width  of  the  pentagonal  display  used  in  ILIOS-type  systems. 
Although  the  target  raster  field  of  view  is  very  small,  no  known  projection  technique  capa¬ 
ble  of  real  time  television  presentation  had  the  resolving  power  to  provide  a  modulation 
of  20%  at  this  resolution. 

SYSTEM  DEFINITION  STUDY  • 

A  prime  contract  was  awarded  to  General  Electric  by  AFHRL  to  implement  the  2363  sys¬ 
tem  in  the  fall  of  1979.  Hughes  was  awarded  a  subcontract  by  GE  to  participate  in  a 
"Phase  I"  system  definition  effort  to  define  an  additive-color  liquid  crystal  light  valve 
projector  capable  of  meeting  the  minimum  requirements  of  the  program  and  of  ultimately 
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meeting  the  design  goal  levels  of  performance 
(see  Figure  2).  Two  candidate  projector  con¬ 
figurations  were  analyzed  during  the  study. 

Both  of  them  combined  three  liquid  crystal 
light  valve  (LCLV)  image  channels  in  an  addi¬ 
tive  color  configuration;  however,  the  LCLV 
image  channels  were  implemented  in  different 
ways.  In  the  first,  a  high  resolution  5"  CRT 
was  used  to  drive  a  2"  LCLV  through  demagni- 
fying  optics  (see  Figure  3(a).  In  the  second,  a 
larger  3"  fiber  optic  LCLV  was  exposed  by  a 
fiber-optic  faceplate  CRT  with  which  it  was  in 
direct  contact.  The  former  configuration  can  - 
with  the  demagnifying  optics  -  provide  a  very 
small  CRT  line  width  to  drive  the  LCLV. 
However,  it  was  found  during  the  study  that  it 
was  not  feasible  to  implement  a  dual-beam 
CRT  (the  dual  beams  were  required  to  meet  the 
high  writing  rate  requirements  at  60  Hz  field 
rates)  capable  of  simultaneously  meeting  the 
very  high  target  raster  resolution  and  the  very 
high  background  brightness  levels  (needed  be¬ 
cause  of  the  inefficiency  of  the  demagnifying 
optics). 


2700-LINE  (2 *  OF  ARC) 
TARGET  MINI-RASTERS 


60  Hi  REPEAT  FIELD 
FULL  COLOR 


Figure  1.  Baseline  Requirements  for  Project 
2363.  The  requirements  shown  represent  a 
significant  challenge  to  the  Visual  System. 


[increase  size?; 
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Figure  2.  High  Performance  2363  —  Compliant  Projector  Configuration.  The  key  to  meeting 
target  raster  resolution,  the  critical  parameter,  is  increasing  the  light  valve  prism  and  CRT  size 
to  3  inches.  High-speed  CRT  drive  circuitry  and  sophisticated  digital  circuits,  driven  by  stored 
correction  values,  ensure  that  high  resolution  and  color  convergence  are  maintained  over  the 
full  displayable  area. 
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(a)  LENS  COUPLED  CRT/LV  (b)  FIBER  OPTIC  COUPLED 

CHANNEL  CRT/LV  CHANNEL 


Figure  3.  Comparison  of  Fiber-Optic  and  Lens-Coupled  CRT/Light  Valve 
Image  Channels.  The  former  is  selected  for  the  2363  projector  even 
though  it  requires  a  3”  light  valve,  because  it  is  more  compact,  requires 
less  brightness  because  of  the  higher  optical  coupling  efficiency  and  does 
not  require  the  development  of  a  high  risk  dual  beam  CRT. 


Furthermore,  this  approach  demanded  higher  resolution  from  the  light  valve  than 
was  available  and  also  resulted  in  a  projector  plate  too  large  and  heavy  to  be  mounted  on  a 
dodecahedron  structure.  The  second  projector  approach  (Figure  3(b)),  although  limited 
in  resolution  by  the  CRT  line  width,  was  nevertheless  found  to  have  the  requisite  projec¬ 
tion  package  —  based  on  the  assumption  that  increasing  the  size  of  the  light  valve,  the 
polarizing  prism,  and  the  fiber  optic  CRT  and  increasing  the  resolution  of  the  latter 
was  indeed  feasible.  Although  there  was  significant  risk  in  developing  these  components, 
these  risks  were  judged  to  be  lower  than  for  a  dual  beam  CRT,  and  the  lower  weight 
and  size  were  judged  to  be  a  considerable  advantage.  For  these  reasons,  the  second 
approach  was  selected  as  the  baseline  approach  to  meeting  2363  requirements. 

An  overview  of  the  implementation  of  this  3”  LCLV  projector  developed  during  the  Phase  I 
study  is  given  in  Figure  2.  The  figure  highlights  the  differences  between  this  projector 
and  the  full  color  projector  developed  under  Project  1958.  These  differences  are  attribu¬ 
table  to  the  need  to  meet  the  stringent  requirements  of  resolution,  color  convergence 
(at  the  high  resolution),  raster  insetting  (without  detecting  raster  outline)  for  three 
targets,  noninterlaced  60  Hz  operation  and  increased  light  output.  Specific  component 
features  and  hardware  implementation  techniques  to  meet  these  requirements  are  briefly 
summarized  below. 

•  Resolution.  Increase  "usable  diameter"  of  light  valve  and  CRT  by  a  factor  of 
3. 0"/l.  8"  =  1.67  (note:  2"  light  valve  has  usable  diameter  of  only  1.8").  As  a 
consequence,  increase  diameter  of  all  appropriate  illumination  and  projection 
optics  components  as  required.  Increase  resolution  of  CRT  by  reducing  the 
line-width  from  1. 1  mil  to  0.6  mil.  Match  or  exceed  upper  20  percentile 
resolution  of  LCLV  (50%  square  wave  MTF  at  20  cycles/mm).  Curve  inside 
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(phosphor)  surface  of  CRT  fiber  optic  faceplate,  and  provide  digitally  mapped 
focus  and  astigmatism  programming  to  minimize  loss  of  resolution  across  the 
field  of  view, 

•  Color  Convergence.  Provide  novel  color  convergence  technique  based  on  digi- 
tally  mapped  and  stored  correction  points.  (Note;  a  variation  of  this  technique 
has  since  been  incorporated  in  a  two-channel  liquid  crystal  projector.)  Use 
stable  deflection  circuits  and  provide  optical  feedback  off  screen  to  correct  for 
residual  position  and  gain  drift.  Provide  microprocessors-aided  interpolation  to 
assist  operator  with  rapid  convergence  alignment  of  the  whole  display  area. 
Provide  sufficient  dynamic  range  on  color  convergence  circuitry,  so  that  the 
techniques  implemented  to  achieve  convergence  can  be  used  to  linearize  the 
display  over  the  full  field  of  view  and  to  edge  match  with  adjacent  windows. 

•  Inset  Raster.  Provide  precision  linear  defies  don  circuits  to  accurately  position 
target  rasters.  Provide  specially  shie’ued  magnetics  on  CRTs  to  minimize  mag¬ 
netic  hysteresis  effects  which  can  cause  tar  get /background  misregistration. 
Provide  capability  to  dither  backgr^u  i*d  raster  lines  to  eliminate  line  structures. 

•  60  Hz  Operation.  Shape  raster  to  be  octagonal  to  eliminate  wasted  trace  time. 
Use  trace-trace  (vs  trace-retrs.ee)  -ax-4-,  of  raster  writing  to  minimize  time 
required  for  retrace,  and  to  provide  the  necessary  compensation  in  the  conver¬ 
gence  circuitry  to  minimize  misalignment  of  adjacent  lines.  Provide  a  100  MHz 
pixel  data  rate  capability,  with  digital  gamma  correction  to  avoid  excessive 
degradation  of  the  video  signa.  due  to  gamma  correction.  Increase  CRT  anode 
voltage  to  22  KV,  and  provide  high  beam  current  capability  to  provide  the  dithered 
highlight  brightness  to  drive  the  light  valve  at  the  high  background  raster  writing 
rates. 

•  Increase  Light  Output.  Increase  lamp  power  to  3. 0  KW  (note:  with  recent 
improvements  in  illumination  system  efficiency  this  should  not  be  necessary). 

Coat  all  lens  surfaces  to  minimize  optical  losses.  Optimize  dichroics  to  achieve 
best  balance  between  efficiency  and  color  range. 

As  shown,  a  number  of  sophisticated  techniques  muBt  be  employed  to  meet  the  specified 
requirements.  Despite  the  apparent  complexity,  design  analysis  during  the  Phase  I 
study  showed  that  these  techniques  could  be  implemented  with  a  reasonable  number  of 
circuits:  18  analog  assemblies  and  60  5"  x  5"  high  density  digital  and  analog  circuit 
boards  for  the  three  channel  projector.  It  was  concluded  that,  assuming  the  three 
critical  components  could  be  developed,  a  4'  of  arc  background,  2'  of  arc  target  60  Hz 
noninterlaced  field  full  color  visual  system  was  entirely  feasible. 

2363  COMPONENT  DEVELOPMENT  PHASE  (CDP) 

To  limit  projector  development  risk,  it  was  decided  to  first  develop  its  three  critical 
components.  This  component  development  phase  (CDP)  of  the  program  was  started  in 
June,  1979  with  the  following  stated  objectives: 

•  To  develop  the  three  components  critical  to  implementing  a  compliant  projector: 
the  3”,  0.6  mil  fiber  optic  CRT,  the  3"  polarizer/analyzer  prism,  and  the  3" 
light  valve. 
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•  To  integrate  these  components  into  a  working  3"  monochrome  breadboard 
projector  and  demonstrate  operation  with  test  patterns  and  television  video. 

•  To  confirm  by  testing  that  the  projector  has  desired  target  and  background 
resolution  under  conditions  simulating  those  in  the  final  projector. 

3  Inch  High  Resolution  0.6  Mil  CRT 

The  cathode  ray  tube  is  one  of  the  important  components  in  the  system  in  terms  of  achiev¬ 
ing  the  desired  projector  resolution.  There  are  three  major  requirements  that  the  CRT 
has  to  meet:  1)  line  width  at  the  low  writing  rate  of  10  Kips  must  be  0. 6  mils  or  better 
for  the  target  raster,  2)  line  width  at  the  high  writing  rate  of  300  Kips  for  the  background 
raster  must  be  less  than  1. 1  mils,  and  3)  the  CRT  useful  screen  size  must  be  3"  or 
greater.  The  challenges  in  designing  this  tube  were  depositing  the  phosphor  to  yield 
the  desired  combination  of  resolution  and  efficiency  for  the  target  and  background  rasters 
respectively,  building  a  gun  capable  of  providing  the  desired  line  widths  for  the  two 
widely  different  raster  writing  rate  conditions,  and  building  a  3"  fiber  optic  CRT  for 
which  the  fiber  optic  quality  was  acceptable  over  the  full  usable  area. 

The  first  two  objectives  were  met  with  unqualified  success.  Development  of  the  phosphor 
was  the  achievement  of  a  major  effort  by  the  CRT  supplier,  Thomas  Electronics.  A 
number  of  phosphor  types,  weights  (particle  sizes  )  and  layer  thicknesses  were  tried 
before  the  final  formula  was  found  for  meeting  the  resolution  and  efficiency  requirements. 
It  was  concluded  that  PI  phosphor  is  the  best,  that  line  width  as  low  as  0. 5  mils  is 
feasible,  and  that  the  resolution  is  primarily  limited  by  phosphor  particle  size,  as 
opposed  to  CRT  gun  limitations.  In  the  design  of  the  gun  the  challenge  was  to  simultan¬ 
eously  obtain  high  resolution  at  low  writing  rates  and  a  medium  high  resolution  at  very 
high  writing  rates,  in  high  beam  currents.  These  conflicting  objectives  were  achieved: 
a  50%  linewidth  of  0.  5  mils  (20%  better  than  required)  was  measured  at  10  Kips,  and  a 
line  width  of  0.9  mils  (-20%  better  than  required)  was  measured  at  300  Kips.  Several 
iterations  on  tube  design  were  required  to  achieve  these  results.  Two  deliverable  assem¬ 
blies  were  built  by  Thomas.  The  first  one,  with  a  4"  diameter  faceplate,  was  delivered 
in  July,  the  second,  with  a  2"  diameter  faceplate,  was  delivered  in  late  October  (see 
Figure  4).  The  latter  exhibited  the  required  performance.  The  two  inch  faceplate  was 
selected  for  the  second  assembly  in  order  to  obtain  the  desired  fiber  pptic  faceplate 
quality.  However,  one  can  confidently  extrapolate  performance  of  this  2"  CRT  up  to 
the  3"  size.  The  magnetics  (identical  for  the  two  tubes)  on  both  CRTs  were  excellent, 
with  both  exhibiting  a  falloff  in  resolution  of  less  than  15%  over  the  usable  area. 

A  problem  was  encountered  with  the  fiber  optic  faceplate.  The  original  supplier  of 
quality  fiber  optic  faceplates  went  out  of  business  and  was  bought  out  by  another  fiber 
optic  house.  The  latter  company  was  unable  to  transfer  the  technology  for  making  the 
very  high  quality,  chicken-wire  free  numerical  aperture  1.0  face  plates,  and  conse¬ 
quently  was  unable  to  meet  Hughes'  requirements  in  the  first  CRT  assembly.  Thus, 
a  2"  faceplate,  available  from  a  previous  order,  was  used  to  assemble  the  second  CRT, 
which  is  currently  being  used  in  the  2363  breadtx>ard  projector.  The  one  remaining 
issue  in  the  CRT  development  area  then  is  to  find  a  vendor  capable  of  supplying  the  high 
quality  fiber  optic  plates  which  do  not  exhibit  chickenwire  and  can  be  supplied  in  the 
3”  size.  Discussions  with  Galileo  and  Incom  reveal  this  should  be  possible  with  some 
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Figure  4.  2363  CRT  and  Magnetic  Component  Layout.  The  magnetics  are  potted  in  place 
around  the  CRT  (the  high  permeability  magnetic  shield  is  not  shown).  A  2”  version  of 
this  design  has  exceeded  both  target  and  background  resolution  goals  and  represents  a 
significant  advance  in  the  state-of-the-art  of  light-valve  driving  CRTs. 


development  effort,  assuming  that  the  numerical  aperture  can  be  reduced  from  1.0  to 
0.66.  Given  the  sensitivity  of  the  SiLV,  this  is  feasible.  In  fact,  experimental  MTF 
test  results  showed  that  a  reduced  numerical  aperture  is  desirable,  because  it 
reduces  the  criticality  of  maintaining  very  close  contact  between  the  CRT  and  the 
light  valve. 

In  summary,  the  final  CRT  meets  or  exceeds  all  requirements  except  for  size.  It 
has  better  than  required  resolution  up  to  writing  rates  of  30  Kips  (new  IG  requirement), 
good  resolution  uniformity,  and  provides  the  required  brightness  for  all  operating 
modes.  It  was  also  found  that  achieving  this  resolution  is  a  low  yield  process,  and  it 
is  felt  that  attempts  to  further  improve  linewidths  are  probably  not  cost  effective.  The 
one  remaining  problem  -  quality  3"  fiber  optic  faceplates  -  appears  to  be  tractable. 

Index  Oil  Prism.  Two  parallel  approaches  to  implementing  a  3"  prism  were  investi¬ 
gated:  a  fused  silica  prism  and  an  "index  oil"  prism.  Of  the  two,  the  latter  emerged 
as  the  clear  choice  (see  Figure  5).  When  used  with  a  prepolarizer,  it  provides  very 
high  performance,  whereas  the  fused  silica  prism  is  marginal  at  best.  In  addition, 
the  index  oil  prism  is  not  burdened  with  the  single  source  procurement  and  delivery¬ 
time  problems  which  plagued  the  fused  silica  prism. 
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Figure  5.  The  3-inch,  58°  Oil  Polarizer/Analyzer  Prism.  In  conjunction  with 
a  prepolarizer  of  the  same  type,  an  extinction  ratio  of  over  1100  was  obtained. 
The  index  oil  approach  was  selected  over  the  solid  glass  fused  silica  prism  be¬ 
cause  of  its  excellent  performance,  lower  cost,  and  shorter  procurement  lead  time. 


Using  two  identical  58°  index  oil  prisms  in  a  prepolarizer  -  polarizer  combination,  a 
contrast  ratio  of  >1100:1  was  demonstrated.  The  prism  seemed  to  be  unaffected  by  the 
heat  of  the  2500  lumen  (at  the  output)  illuminating  beam,  and  did  not  seem  to  be 
affected  by  vibration. 

The  two  prisms  in  the  2363  breadboard  occupy  a  considerable  volume.  This  is  attributable 
to  three  factors.  First,  a  58°  angle  was  chosen  to  maximize  contrast  ratio  and  to  permit 
comparison  of  the  index  oil  prism's  performance  with  that  of  the  2"  fused  silica  prism, 
which  had  been  built  at  58°.  Second,  prism  size  was  made  large  enough  to  accommodate 
any  conceivable  cone  angle  of  the  illuminant  beam.  Third,  separate  prepolarizer  and 
polarizer  prisms  were  used.  In  a  productized  design,  prism  angle  will  be  reduced  to 
48°,  which  will  reduce  size  but  still  yield  a  contrast  ratio  of  greater  than  500:1.  Size 
will  be  tailored  to  a  reasonable  cone  angle  illumination  system,  and  the  prepolarizer 
and  polarizer  functions  will  be  combined  into  a  single  prism  assembly.  Implementing 
these  three  changes  should  yield  a  volume  reduction  factor  of  greater  than  x  2.5. 

3"  CdS  Light  Valve.  Before  discussing  the  results  of  the  3"  CdS  component  effort,  it 
is  appropriate  to  discuss  a  decision  (later  reversed)  made  at  the  start  of  the  CDP 
effort  which  had  a  significant  impact  on  this  phase  of  the  Hughes  2363  program. 
Determining  the  course  of  action  on  the  CRT  and  the  polarizing  prism  was  straight¬ 
forward.  However,  the  light  valve  posed  the  following  dilemma:  The  two  major 
requirements  imposed  by  the  light  valve  are  increasing  size  to  3"  on  the  one  hand  and 
improving  time  response  of  the  light  valve  on  the  other.  It  was  clear  that  the  Cadmium 
Sulfide  Light  Valve  (CdSLV)  in  production  at  Hughes  Carlsbad  would  not  be  able  to 
meet  the  better  than  20  ms  response  required  to  support  a  60  Hz  refresh  system.  The 
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silicon  photoconductor  light  valve  (SiLV)  was  the  obvious  candidate  to  solve  this  short¬ 
coming.  But  at  the  start  of  this  phase  of  the  program  in  June,  1979,  the  SiLV  was  still 
very  much  an  R&D  device:  it  had  not  yet  been  built  in  the  2"  size,  it  had  demonstrated 
resolution  only  half  as  good  as  that  required,  it  had  exhibited  severe  edge  breakdown 
problems,  it  had  marginal  contrast,  and  it  had  not  yet  been  operated  in  a  fiber-optic 
CRT  driver  mode  with  dynamic  television  imagery.  For  that  matter,  its  response 
time  —  limited  by  the  liquid  crystal  material  —  was  only  marginally  adequate  for  30  Hz 
and  not  good  enough  for  60  Hz  operation.  Consequently,  it  was  decided  to  opt  for 
developing  a  3"  CdS  LV  to  prove  the  feasibility  of  a  3"  size  light  valve  on  the  one  hand, 
and  to  provide  a  means  of  implementing  a  3"  breadboard  system  as  a  way  of  testing 
the  other  components  on  the  other.  A  year  later,  when  the  program  was  reviewed, 
sufficient  work  had  been  done  to  establish  feasibility  of  building  a  3"  CdSLV.  Meanwhile 
the  SiLV  had  made  substantial  progress  during  the  previous  year,  so  that  it  appeared 
close  to  being  operable  in  a  high  light  output  projector.  It  was  therefore  decided  to 
discontinue  the  3'  CdSLV  effort  and  to  devote  the  remaining  funds  to  support  the  SiLV 
effort  at  the  Hughes  Malibu  Research  Laboratories  (where  an  ongoing  SiLV  effort  had 
been  funded  by  AFHRL  on  a  separate  2363-associated  R&D  contract). 

In  view  of  these  facts,  the  subsequent  discussion  in  this  section  will  cover  the  3"  CdS 
LV  and  will  describe  the  breadboard  projector  built  to  integrate  it  with  the  other  two 
CDP-dev eloped  components.  The  following  section  will  cover  in  detail  the  SiLV,  the 
key  element  in  applying  liquid  crystal  projectors  to  simulation  visuals. 

The  results  of  the  3"  CdSLV  effort  were  impressive,  considering  that  only  two  3"  light 
valves  were  constructed  (ten  had  been  planned):  the  second  of  these  light  valves  showed 
surprisingly  good  performance.  Although  it  fell  short  of  the  specification  on  resolution 
and  brightness  uniformity,  it  did  demonstrate  that  no  fundamental  problems  arose 
in  going  to  the  3"  size.  Relevant  to  the  SiLV,  it  was  found  that  the  0.  5"  f/o  plate 
and  the  0.  5"  glass  counter  electrode  plate  were  sufficiently  rigid  to  maintain  a 
uniform  3.  5^  liquid  crystal  thickness  across  the  3"  light  valve.  Equally  relevant 
to  the  SiLV,  much  of  the  tooling  developed  for  the  3"  CdSLV  is  expected  to  be  usable 
with  the  SiLV  when  its  dimensions  are  increased  to  3". 


The  3"  CdSLV  was  installed  in  the  breadboard  projector  and  driven  by  the  3"  (useful) 
screen)  fiber  optic  CRT  to  demonstrate  a  full  3"  system  to  AFHRL  and  General  Electric 
at  the  August,  1980  engineering  design  review.  Because  of  the  poor  quality  of  the  CRT 
fiber  optics  and  line  width,  only  a  limited  number  of  measurements  were  taken  using 
either  limiting  TV  resolution  or  shrinking  raster  technique  (<15%)  over  the  usable  area. 

Breadboard  Projector.  The  breadboard  projector  integrates  the  critical  components 
developed  on  the  Component  Development  Phase  (CDF)  with  appropriate  optics  and  elec¬ 
tronics  to  demonstrate  that  system  resolution  goals  have  been  attained  for  both  the  target 
and  background  rasters. 

The  breadboard  projector  was  laid  out  on  an  optical  plate  to  permit  easy  access  to  all 
components  (see  Figure  6).  Part  of  the  projector  is  the  group  of  three  critical  components: 
fiber  optic  CRT,  the  light  valve,  and  the  prepolarizing  and  polarizing  index  oil  prism.  The 
CRT  is  driven  by  medium  speed  1000-line  electronics  using  linear  amplifiers.  Various 
raster  modes  selected  at  the  control  panel  are  provided  to  realistically  simulate  the  target 
and  background  raster  operating  conditions  of  the  final  system.  Power  supplies  are 
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Figure  6.  Optical  Layout  of  Breadboard  Projector.  The  projector  combines  the  newly 
developed  components  to  generate,  using  the  3  inch  CdS  light  valve,  a  projected  display 
having  a  limiting  resolution  of  well  over  3000  TV  lines  (extrapolating  target  resolution 
to  width  of  projected  pentagonal  image). 


mounted  in  a  rack  remote  from  the  optical  plate.  Illumination  is  supplied  by  a  3.  0  KW 
xenon  arc  lamp  assembly  featuring  a  nutated  elliptical  reflector  which  is  housed  in  an  off- 
the-shelf  commercial  lamp  house,  with  the  light  collimated  by  the  two  lenses  of  an  illumi¬ 
nating  optical  system  Cold  mirrors  remove  the  heat  from  the  beam.  The  prepolarizing 
and  the  polarizing  index  oil  prisms  further  filter  the  IR  energy  and  yield  a  very  high 
extinction  ratio  which  eliminates  the  prism  as  a  factor  affecting  the  system  contrast  ratio. 
A  custom  designed  3"  telecentric  projection  lens  images  the  light  valve  generated  polar¬ 
ized  pattern  on  a  60"  diameter  round  screen  at  a  distance  of  20  feet.  The  lens  provides 
modulation  in  excess  of  90%  at  the  spatial  frequency  corresponding  to  two  minutes  of  arc. 
These  optical  elements  combine  to  project  2500  open  gate  lumens  with  a  brightness  non¬ 
uniformity  of  less  than  15%  over  the  quality  circle. 


To  permit  the  use  of  off-the-shelf  electronics  in  order  to  minimize  the  cost  of  building  the 
breadboard  projector,  it  was  decided  to  measure  the  background  raster  (under  realistic 
writing  rate  conditions)  using  the  shrinking  raster  method.  Vertical  and  horizontal 
rasters  to  measure  either  vertical  or  horizontal  resolution  are  provided.  The  target 
raster  is  generated  by  a  standard  525  line  TV  input,  with  the  raster  size  variable  to 
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change  the  effective  resolution  from  approximately  0.5  min  of  arc  to  3  min  of  arc,  and 
with  the  raster  positionable  anywhere  on  the  displayable  area  to  verify  resolution 
uniformity. 

The  breadboard  projector  was  operated  in  two  modes.  Initially,  the  first  delivered  4" 

CRT  was  used  to  drive  a  3"  CdSLV.  As  a  result  of  the  shift  to  the  SiLV,  and  because 
the  3"  fiber  optic  CRT  faceplates  were  not  available  without  chickenwire  artifacts,  the 
final  projected  configuration  featured  the  very  high  quality  2"  fiber  optic  CRT  driving 
either  a  2"  SiLV  or  CdSLV. 

This  breadboard  system  was  used  to  determine  the  projector  resolution  obtainable  with 
a  3"  liquid  crystal  light  valve  system,  using  either  a  CdS  or  a  silicon  light  valve. 

The  results  of  these  tests  are  summarized  in  the  section  following  the  one  below  which 
describes  the  construction  and  performance  of  the  SiLV. 

DESCRIPTION  OF  THE  SILICON  PHOTOCONDUCTOR  LIQUID  CRYSTAL  LIGHT  VALVE 

The  key  to  the  successful  application  of  liquid  crystal  light  valve  technology  to  simulation 
is  the  Silicon  Photoconductor  Light  Valve  (SiLV).  Although  attempts  to  improve  time- 
response  characteristics  of  the  Cadmium-Sulfide  Photoconductor  Light  Valve  (CdSLV), 
currently  in  production  to  support  a  growing  line  of  graphic  large  screen  display  pro¬ 
ducts,  resulted  in  significant  improvements,  achieving  the  fast  response  time  required 
to  meet  the  needs  of  simulation  does  not  appear  practical  with  this  light  valve.  For  this 
and  other  reasons  Hughes  decided  to  investigate  the  feasibility  of  a  silicon  photoconductor- 
based  light  valve.  Having  established  its  feasibility  in  1976,  an  IR&D  program  was 
initiated  to  develop  a  device  which  gives  not  only  better  time-response  but  also  promises 
higher  sensitivity,  contrast  ratio  and  resolution.  With  the  Hughes  IR&D  program  comple¬ 
mented  by  support  from  AFHRL  under  Projects  1958  and  2363  and  additional  support 
from  NTEC,  NAVSEA  and  PM-TRADE,  development  of  the  device  has  progressed 
rapidly  over  the  past  two  years.  To  illusti'ate  this  progress,  Figure  7  shows  the 
improvement  in  the  limiting  resolution  obtained  with  SiLVs  over  this  span  of  time. 

Today  the  basic  SiLV  configuration  is  well  established  and,  except  for  some  device 
engineering  issues  to  be  resolved,  the  light  valve  can  provide  performance  (although 
not  cosmetic  quality)  comparable  to  that  of  the  CdSLV,  and  with  much  better  time 
response.  Consistently  good  cosmetic  quality  will  be  achieved  when  the  light  valve  is 
transferred  into  a  production  environment. 

To  gain  historical  perspective  and  an  appreciation  for  its  comparative  advantages,  it  is 
helpful  to  view  the  SiLV  as  an  evolution  in  the  state  of  the  liquid  crystal  light  valve  art. 

The  generic  advantages  of  a  liquid  light  valve  as  a  projection  technique  are  numerous, 
with  three  of  these  particularly  relevant  to  simulation  visuals.  First,  the  light  valve 
modulates  ("valves")  an  externally  generated  light  beam  with  a  liquid  crystal  layer  which 
does  not  absorb  light  but  simply  changes  its  polarization.  High  light  output  levels  are 
therefore  achievable.  Second,  the  technique  is  not  constrained  to  a  raster-only  mode  of 
operation,  a  feature  which  is  very  valuable  in  a  variety  of  simulation  applications.  Finally, 
the  light  valve  is  photo-activated  (as  opposed  to  being  modulated  by  an  electrostatic 
charge  as  in  most  other  light  valves),  which  results  in  separating  the  light  valve  function 
from  the  image  generating  function  and  its  complexities  -  the  latter  function  typically 
being  performed  for  the  photo-activated  light  valve  by  a  high  resolution  fiber  optic 
faceplate  CRT  which  can  be  designed  and  built  independently  of  the  light  valve.  This 
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Figure  7.  Progress  in  Limiting  Resolution  Obtained  with  the  SiLV.  Steady 
increase  in  resolution  is  attributable  to  increasing  the  grid  spatial  frequency, 
(see  Figure  10  for  details),  and  continued  improvements  in  silicon  processing 
and  mirror  deposition,  respectively. 


separation  of  the  image  generation  and  light  valving  functions  is  in  turn  beneficial  for 
several  reasons.  Devices  which  combine  these  two  functic'  3  are  typically  complex,  hard 
to  maintain  and  expensive.  By  comparison  the  LCLV  is  simple,  easy  to  maintain,  and 
should  be  lower  in  cost  once  in  production.  In  addition,  separation  of  the  two  functions 
allows  each  one  to  be  optimized  individually,  thus  providing  a  great  deal  of  flexibility  in 
adapting  the  technique  to  a  variety  of  applications  without  costly  re-engineering  of  a 
complex  device.  The  accomplishments  on  the  2363  program  attest  to  this  flexibility. 

To  introduce  and  describe  the  SiLV,  a  brief  description  of  the  basic  principles  of  oper¬ 
ation  of  the  liquid  crystal  light  valve  is  given,  followed  by  a  comparison  of  the  silicon  and 
CdS  light  valves  and  a  description  of  the  theory  of  silicon  light  valve  operation.  Finally, 
the  performance  levels  obtained  with  SiLVs  built  to  date  are  summarized. 

Description  of  the  Generic  Liquid  Crystal  Light  Valve.  A  simplified  explanation  of  the 
generic  hybrid  field  effect  (liquid  crystal)  light  valve  is  given  here  for  reference;  more 
detailed  description  of  the  device  and  its  operation  can  be  found  in  the  literature.  ^ 

Figure  8  shows  the  construction  of  the  device. 

The  principle  of  operation  is  best  described  by  initially  considering  the  cell  simply  as  a 
photoconductive  layer  and  a  liquid  crystal  layer  of  small  thickness  sandwiched  between 
two  transparent  conductive  surfaces.  With  the  photoconductor  unexposed,  a  bias  voltage 
applied  to  the  two  transparent  conductive  surfaces  appears  directly  across  the  photocon¬ 
ductive  layer,  leaving  the  liquid  crystal  in  its  normal  clear  state.  When  the  photoconductor 
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the  liquid  crystal  light  valve  arc  the  same  lor  the  CdS  and  silicon  photo  con¬ 
ductor  light  valves.  The  two  differ  in  the  photo  conductor  material,  thickness, 
the  type  of  bias  voltage,  and  the  method  of  implementing  the  light  function. 
Note  that  the  figure  is  not  drawn  to  scale;  blocking  the  thickness  of  the  fiber 
optic  and  glass  counter-electrode  plates  is  0.5  inch  (12.5  mm),  whereas  the 
active  layers  arc  approximately  1.5  to  5  mils  (35  to  125  p). 


is  activated  by  light,  its  impedance  drops,  and  the  bias  voltage  is  applied  across  the 
liquid  crystal  layer  causing  the  image  exposing  the  photoconductor  to  be  reproduced  as 
a  birefringent  pattern  in  the  liquid  crystal  light  valve.  To  project  this  liquid  crystal 
image,  a  dielectric  mirror  is  inserted  directly  behind  the  liquid  crystal  layer.  The 
cell  is  illuminated  with  near-collimated  polarized  light  which  travels  through  the  liquid 
crystal  layer,  is  then  reflected  by  the  dielectric  mirror,  and  passes  through  liquid 
crystal  a  second  time  before  being  projected. 

In  the  areas  where  the  liquid  crystal  material  has  been  activated  by  the  voltage  which 
varies  with  light  input,  the  liquid  crystal  birefringence  causes  a  rotation  of  the  axis  of 
polarization,  which  then  permits  the  light  to  pass  through  a  (prism)  analyzer  to  the 
screen.  To  prevent  the  polarized  projection  light  from  exciting  the  photoconductive 
layer,  an  opaque,  light-blocking  layer  may  be  inserted  between  the  dielectric  mirror 
and  the  photoconductive  layer.  This  stack  of  thin  film  layers  (i.e. ,  the  two  transparent 
conductive  layers,  the  photoconductor,  the  light  blocking  layer  if  required,  the  mirror 
and  the  liquid  crystal  layer)  is  sandwiched  between  a  12. 5  mm  thick  glass  plate  on 
the  projection  side  and  a  12. 5  mm  glass  or  fiber  optic  plate  on  the  input  side. 

Comparison  of  CdS  and  Silicon  Light  Valves.  Figure  9  graphically  illustrates  the 
differences  between  the  two  light  valves.  The  silicon  photoconductor  is  a  thin  wafer 
(5  mil)  as  opposed  to  the  CdS  thin  film  layer;  it  is  much  thicker  (by  a  factor  of  five) 
than  the  CdS.  The  lower  capacitance  and  higher  capacitive  impedance  associated  with 
the  increased  thickness  result  in  the  ability  to  swing  a  wider  range  of  voltages  across 
the  liquid  crystal  and  therefore  yield  a  higher  contrast  ratio.  The  silicon,  instead  of 


210 


BACK 

CONTACT 

PHOTO¬ 

CONDUCTOR 

\ 

LIGHT 

BLOCKING 

LAYER 

MIRROR 

CdS  LV 

ITO 

POLY  CdS 

CdTe 

Si02/TI02 

Si  LV 

SI 

(P-  DOPED) 

Si  GRID 

SI/SI02 

3  K  I 

1 


■IfeijMMiiljfe M||ii||>>V(y 


♦same  as 
cas  lv 


Figure  9.  Comparison  of  Cadmium  Sulfide  (CdS)  and  Silicon  Photoconductor  Light  Valves. 
The  all-silicon  construction  of  silicon  light  valve  results  in  a  simpler,  more  producible  device 
with  the  potential  for  superior  performance. 


being  sputtered  down  on  the  fiber  optic  plate  as  a  thin  film  layer  as  the  CdS  is,  is  a  thin 
wafer  which  must  be  attached  to  the  plate  and  flattened  to  make  the  liquid  crystal  layer 
of  uniform  thickness.  The  silicon  photoconductor  is  more  sensitive,  and  sensitive 
over  a  wider  range  of  the  spectrum  (approximately  0.  4  to  1. 1  micron),  than  the  CdS. 
While  this  means  that  less  imaging  light  is  required  to  drive  it  (an  advantage),  it  does 
demand  of  the  light  blocking  layer  a  greater  absorption  over  this  wider  spectral  range. 
Fortunately,  it  appears  feasible  to  combine  the  functions  of  light  blocking  and  light 
reflection  in  a  single  Si/SiC>2  mirror  which  simplifies  the  overall  device  structure.  A 
further  difference  between  the  two  light  valves  is  the  back  contact;  this  is  an  ITO  thin 
film  for  the  CdSLV  and  a  heavily  doped  layer  in  the  wafer  for  the  SiLV. 


In  terms  of  overall  device  construction,  the  SiLV  appears  to  be  much  simpler.  Fewer 
layers  are  required  and,  perhaps  more  importantly,  the  number  of  materials  used  is 
drastically  decreased.  This,  coupled  with  the  use  of  a  more  mainstream  material 
(silicon),  should  make  the  SiLV  a  much  more  producible  device. 


There  is  another  significant  difference  between  the  two  devices.  While  the  CdS  photo¬ 
conductor  is  a  continuous  thin  film  which  depends  on  its  high  lateral  impedance  to  reduce 
image  spreading  and  thus  obtain  good  resolution,  in  the  SiLV,  the  surface  effects  at  the 
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MOS  interface  will  cause  the  image  to  diffuse  unless  some  precautions  are  taken. 

Thus,  the  SiLV  requires  a  grid  structure  to  confine  the  photo-generated  charge  pattern; 
this  is  illustrated  in  Figure  10(a).  The  left  side  shows  the  photoelectrons  generated 
close  to  the  back  surface  being  driven  across  the  depleted  silicon  toward  the  right  hand 
surface  where  they  are  collected  in  heavily  doped  islands  ("charge  buckets")  defined 
by  the  grid  structure  shown  in  Figure  10(b).  If  the  field  across  the  silicon  wafer  is 
maintained,  the  exposing  image  will  generate  a  corresponding  charge  pattern  at  the 
right  hand  (mirror)  side  of  the  wafer,  subject  to  some  degradation  because  of  the  charge 
averaging  action  of  the  charge  buckets.  To  keep  the  buckets  from  filling  up  and  spilling 
over  (thus  causing  the  image  generated  charge  pattern  to  wash  out),  the  voltage  is 
periodically  briefly  reversed  (accumulation  phase,  see  below)  to  empty  out  the  buckets 
and  start  the  image  generation  cycle  over  again.  Because  of  the  above,  the  SiLV 
requires  a  pulsed  biasing  voltage  instead  of  the  sinewave  voltage  used  by  the  CdSLV, 

In  summary,  the  basic  operation  of  the  SiLV  requires  a  grid  structure  which  limits  the 
resolution  performance  of  the  final  device.  Three  grid  structures  of  increasing  reso¬ 
lutions  have  been  implemented  to  date:  30,  50  and  100  grid  lines/mm.  The  bucket 
sizes  and  spacings  corresponding  to  these  frequencies  are  shown  in  Figure  10(b).  In 
theory,  150  grid  lines/mm  should  be  feasible;  in  practice,  the  best  limiting  resolution 
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Figure  10.  Illustration  of  Image  Resolving  Mechanism  in  the  SiLV.  Photoelectrons  generated  by 
light  absorption  near  the  back  contact  travel  across  the  depleted  silicon  wafer  and  are  collected  in 
the  heavily  doped  square  regions  which  act  as  “charge  buckets”  to  store  the  image  temporarily. 
Spatial  frequency  of  the  grid  lines  which  define  the  buckets  is  the  primary  factor  in  determining 
light  valve  resolution. 


to  date  with  the  100  grid  lines/mm  has  been  70  lines/mm.  For  this  case  the  grid  pat¬ 
tern  is  no  longer  observable  in  the  projected  image  (thus  no  aliasing  occurs) ,  indicating 
that  a  mechanism  other  than  the  grid  structure  limits  device  resolution. 

Advantages  of  the  SiLV  over  its  CdS  counterpart  are  briefly  summarized  as  follows: 

•  Higher  sensitivity  due  to  silicon  photoconductor. 

•  Faster  response  (limited  by  the  liquid  crystal)  due  to  silicon  photoconductor. 

•  Higher  contrast  ratio/brightness  due  to  thicker  photoconductor /higher 
switching  value. 

•  Simpler  device  construction,  with  potential  for  better  manufacturability. 

•  Potential  for  higher  resolution  -  if  theoretical  limits  are  realized  with  improved 
processes. 

SiLV  Theory  of  Operation.  The  SiLV  is  based  on  a  thin  wafer  of  high  resistivity,  single 
crystal  silicon  as  the  photoconductor.  The  device  consists  of  the  silicon  photoconductor 
coupled  with  an  oxide  layer  to  form  a  MOS  structure.  The  readout  beam  is  retro- 
reflected  by  means  of  a  dielectric  Si/SiC>2  mirror  through  the  liquid  crystal  operating  in 
the  hybrid  field  effect  mode.  As  mentioned  above,  this  mirror  also  acts  as  a  light 
blocking  layer  which  serves  to  optically  isolate  the  photoconductor  from  the  high  intensity 
readout  beam.  The  MOS  mode  of  operation  consists  of  periodic  depletion  and  accumu¬ 
lation  phases.  In  the  depletion  (active)  phase,  the  high  resistivity  jr-silicon  is  depleted 
completely,  and  electron  hole  pairs  generated  by  the  input  light  are  swept  by  the  electric 
field,  thereby  producing  the  signal  current  that  activates  the  liquid  crystal.  The  signal 
electrons  residing  at  the  Si/Si02  interface  are  recombined  by  pulsing  the  MOS  into  a 
short  accumulation  phase.  The  spatial  resolution  of  the  input  image  across  the  silicon 
is  retained  by  means  of  a  boron-implated  p-grid  at  the  7r-Si/Si02  interface.  The  grid 
acts  to  focus  the  incoming  electrons  into  the  resolution  cell  defined  by  it,  as  well  as  to 
form  charge  "buckets"  of  the  electrons  already  residing  at  the  Si/  Si02-  This  prevents 
lateral  "spill-over"  and  consequent  smearing  of  the  charge  pattern.  A  jh-  (boron)  ring  is 
diffused  at  the  silicon  edge  to  prevent  peripheral  minority  carrier  injection  into  the 
active  region.  A  thin  layer  of  fast  response,  positive  anisotropy,  liquid  crystal  with  a 
45°  twist  angle  is  employed  as  the  light  modulator.  The  input  electrode  consists  of  a 
fiber-optic  faceplate  coated  with  indium  tin  oxide.  A  liquid  crystal  spacer  as  well  as 
an  indium  tin  oxide  conductive  layer  are  deposited  on  the  front  glass  electrode.  The 
whole  structure  is  assembled  within  an  air  tight  anodized  aluminum  holder  as  shown  in 
Figure  11,  which  also  shows  a  photograph  of  the  finished  light  valve.  The  light  valves 
produced  feature  a  1.6"  clear  aperture. 

SiLV  Performance.  The  table  summarizes  the  significant  performance  levels  of  SiLVs 
built  to  date  and  compares  them  with  the  requirements  of  Project  2363.  Figure  12 
compares  the  MTFs  for  the  highest  MTF  SILVS  with  two  CdSLVs,  one  of  them  a  repre¬ 
sentative  sample  and  the  other  one  of  the  highest  resolution  CdSLVs  built. 
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Figure  11.  Hardware  Characteristics  of  the  Liquid  Crystal  Light  Valve.  The  light  valve  is 
a  compact  (3.0”  diameter,  1.5”  thick),  rugged  device.  In  the  CdS  light  valve,  the  CdS  photo¬ 
conductor  is  a  thin  film  layer  sputtered  on  the  fiber  optic  plate;  in  the  SiLV  the  photocon¬ 
ductor  is  a  5  mils  thick  silicon  wafer  which  is  independently  processed  and  is  then  attached 
to  the  fiber  optic  plate. 

As  shown  in  the  table ,  the  ability  to  meet  or  exceed  the  required  performance  levels  has 
been  clearly  demonstrated  in  the  areas  of  contrast  ratio  (although  at  low  light  levels  only), 
sensitivity  and  response  time.  Resolution  to  meet  the  requirements  of  a  2'  of  arc 
(2700  TVL/width  at  20  percent  MTF)  has  not  been  achieved;  when  the  grid  density  was 
increased  from  50  to  100  grid  lines/mm,  the  light  valve  MTF  did  improve  at  the  higher 
spatial  frequencies  (limiting  resolution  went  from  50  to  70  TVL/mm)  but  did  not  improve 
modulation  at  the  spatial  frequency  corresponding  to  2'  of  arc  (38  TV  lines/mm).  Further 
progress  in  exploiting  the  100  grid  lines/mm  technology  is  expected  to  match  the  best 
CdSLV  resolution  (sufficient  for  2'  of  arc)  in  the  near  term,  and  is  also  expected  to 
approach  the  1. 5'  of  arc  capability  in  the  long  run.  Although  performance  falls  short  in 
three  other  areas  (contrast  ratio,  switching  ratio  and  maximum  light-blocking  level), 
these  are  degraded  by  a  Si/SiC>2  mirror  with  inadequate  light-blocking  capability.  This 
shortcoming  can  be  corrected  by  an  improved  mirror  design.  A  further  shortcoming 
at  the  time  of  this  writing  is  poor  uniformity  due  to  the  fact  that  no  effort  had  yet  been 
made  to  attach  the  wafer  to  the  fiber  optic  plate;  the  wafer  was  allowed  to  "float"  freely. 
Implementation  of  a  wafer  attachment  approach  is  expected  to  result  in  a  uniformity  of 
30%  or  better  in  the  near  future.  It  should  be  noted  that  the  best  performance  capabilities 
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Figure  12.  Comparison  of  the  MTFs  of  the  Best  50  and  100  Grid  Lines/mm 
SiLVs  with  a  medium  and  a  high-resolution  CdSLV.  The  silicon  light  valves 
exhibit  resolution  comparable  to  that  of  an  average  CdSLV,  with  the  100  grid 
lines/mm  SiLV  demonstrating  better  limiting  resolution. 

listed  in  the  table  have  not  been  combined  in  a  single  device  as  yet,  although  this 
appears  to  be  primarily  a  matter  of  yield  rather  than  due  to  fundamental  device  limita¬ 
tions.  Hughes  is  confident  that  the  planned  1981  effort  will  meet  all  stated  performance 
requirements.  Further  activity  is  expected  to  confirm  the  feasibility  of  increasing 
the  light  valve  size  to  3  inches. 

RESULTS  OF  BREADBOARD  RESOLUTION  TESTS 


Resolution  tests  were  conducted  for  the  breadboard  projector  using  the  two  CdS  light 
valves  (Nos.  10063  and  10470),  for  which  MTF  curves  are  shown  in  Figure  12,  and 
several  of  the  50  and  100  grid-line/mm  SiLVs  built.  The  projector  MTF  plots  for 
these  two  CdSLVs  and  the  SILVS  yielding  the  best  projector-level  response 
are  shown  in  Figure  13.  All  of  these  tests  were  taken  in  a  projector  configuration  in 
which  both  the  CRT  and  the  light  valves  were  2  inches  in  diameter  for  reasons  pre¬ 
viously  discussed.  First,  for  the  CRT  the  final  2  inch  assembly  provided  the  best 
fiber  optic  quality  and  the  best  line  width.  Second,  the  SiLV  was  available  only  in 
the  2  inch  size.  Finally,  although  a  3”  CdSLV  was  constructed,  its  resolution  was  not 
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TABLE  OF  CURRENT  STATUS  OF  SiLV  PERFORMANCE  CAPABILITIES 


Parameter 

Achieved 

Required  by  2363 

Size 

1. 6"  usable 

3.0" 

Resolution: 

MTF  @  2'  of  arc 

22%  to  31% 

55% 

Contrast  ratio 

5:1  to  100:1(1^2) 

50:1  (100:1  goal) 

Sensitivity 

10  to  30  uw/cm2 

2 

40  uw/cm 

Response  time  (decay  to  10%) 

10  to  40  ms 

20  ms 

Illumination  level 

2000  to  4000  fc<2> 

12,000  fc 

(1)  High  contrast  ratios  (10:1)  and  switching  ratios  obtainable  only  at  low  light 
levels. 

(2)  Requires  redesign  of  mirror  in  SiLV  to  optimize  light  blocking  capability. 

as  high  as  required  (i.  e. ,  as  high  as  that  of  CdS  LV  No.  10063  Figure  12);  projector 
MTF  measurements  were  therefore  made  with  both  the  No.  10063  CdS  LV  and  a  CdSLV 
(No.  10470)  having  MTF  characteristics  approximately  those  of  the  3"  CdSLV. 

The  square  wave  MTF  measurements  shown  in  Figures  12  and  13  were  taken  in  the 
breadboard  projector  using  one  of  the  following  techniques.  To  characterize  the  light 
valves  (see  Figure  12),  a  chromium-glass  Sayce  chart  was  placed  in  close  optical 
contact  with  the  fiber-optic  plate  of  the  light  valve,  and  the  test  chart  was  back-illumi¬ 
nated  by  a  bright  unmodulated  raster  displayed  on  the  CRT.  The  resulting  aerial  image 
in  the  plane  of  the  screen  was  scanned  by  a  0.  25  mil  slit/photodiode  assembly 
mounted  on  a  motorized  translation  stage,  with  the  amplified  photodiode  output  recorded 
on  an  XY  recorder  driver  horizontally  in  synchronism  with  the  translation  stage.  The 
relative  modulation  was  then  derived  by  dividing  the  amplitude  of  the  nearly  sinusoidal 
output  at  a  given  frequency  by  the  low  frequency  (2.  5  lp/mm)  output  amplitude.  For  the 
projector-level  MTF  tests  (Figure  13)  the  projector  was  driven  by  a  Visual  Information 
Inc.  Model  216  Test  Pattern  generator  with  a  series  of  vertical  bar  test  patterns 
(corresponding  to  various  spatial  frequencies),  and  the  resulting  images  were  scanned 
by  a  slit-scanner  and  recorded  as  described  above.  The  525-line  raster  displayed  on 
the  CRT  was  sized  to  simulate  the  target  raster  writing  rate  conditions  encountered  in 
the  ultimate  full-color  projector  (10,000  inches/sec).  It  should  be  noted  that  SiLV 
MTF  measurements  were  taken  under  low  light  level  conditions. 
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Figure  13.  Comparison  of  Projector  Performance  Using  Three  Light  Valves. 
System  requirements  (35%  MTF  at  38  TVL/mm)  are  nearly  met  with  the 
No.  10063  CdS  LV. 


The  resulting  data  is  shown  in  Figure  13  with  square  wave  modulation  (MTF)  plotted 
against  TV  lines/mm  (spatial  frequency).  Also  shown  along  the  abscissa  are  the  TV 
lines  across  the  pentagon  inscribed  in  a  3.0  inch  diameter  light  valve,  and  the  minutes 
of  arc  at  the  pilot’s  eye  assuming  the  display  covers  a  90°  field  of  view  corresponding 
to  these  spatial  frequency  values.  As  shown,  the  projector  modulation  at  2'  of  arc 
(the  required  2363  target  resolution)  is  29%,  10%  and  6%  for  the  No.  10063  CdSLV,  the 
best  (No.  697)  SiLV,  and  the  No.  10470  CdSLV,  respectively. 

The  system  requirement  of  20%  MTF  at  the  pilot's  eye  translates  into  a  requirement 
of  32%  at  the  output  of  the  breadboard  projector,  based  on  an  educated  estimate  of  the 
pancake  window  and  (final)  projection  optics  MTFs,  and  the  known  resolution  of  the 
breadboard  projection  lens.  Thus,  the  breadboard  projector  with  a  high  resolution 
CdSLV  (No.  10063)  extrapolated  to  3  inches  in  diameter  nearly  meets  the  required 
MTF  of  32%,  and  provides  respectable  MTF  performance  (10%)  at  1.5*  of  arc.  The 
silicon  light  valve  does  not  measure  up  to  performance,  although  it  still  provides  good 
resolution  performance  (note  that  the  SiLV  performance  measured  corresponds  to  a 
30%  MTF  for  1000  TV  lines  across  a  rectangular  display).  As  stated  before,  it  is 
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anticipated  that  exploiting  the  increased  grid  line  frequency  (from  50  to  100  grid  lines/mm) 
will  improve  the  silicon  light  valve  MTF  characteristics  to  match  or  exceed  the  MTF  of 
the  best  CdSLV,  with  an  attendant  improvement  in  total  projector  resolution.  With  this 
improvement  a  system  MTF  of  greater  than  20%  at  2’  of  arc,  and  an  MTF  greater  than 
6%  at  1.5'  of  arc  should  be  realized. 

Although  the  primary  test  effort  emphasis  was  on  characterizing  target  raster  MTF,  a 
number  of  other  tests  were  conducted,  and  are  briefly  summarized.  Shrinking  raster 
measurements  of  background  raster  resolution  as  a  function  of  position  showed  that 
2363  requirements  were  far  exceeded  at  the  high  raster  writing  rates.  Target  resolution 
was  estimated  to  drop  off  less  than  15%  as  a  function  of  display  position  on  the  3  inch  light 
valve.  Finally,  shrinking  raster  measurements  of  target  raster  resolution  as  a  function 
of  CRT  writing  rate — of  interest  because  of  the  possible  desirability  of  painting  larger 
than  3°  x  3°  target  rasters — showed  a  resolution  loss  of  less  than  10%  when  going  from 
10,000  to  50,000  inches  per  second  on  the  CRT. 

CONCLUSIONS 


Significant  progress  has  been  made  on  Project  2363  toward  implementing  a  very  high 
resolution,  multiple-target  capable,  full  color  liquid  crystal  light  valve  projector  for 
in-line  infinity  optics-based  wide  field  of  view  simulator  visuals.  The  early  phase  of 
the  program  investigated  alternative  approaches  to  achieving  the  required  target 
resolution  and  other  demanding  performance  objectives,  and  selected  a  3  inch,  fiber¬ 
optic  coupled  CRT -light  valve  image  generator  as  the  basic  building  block  for  such  a 
projector.  The  study  further  selected  the  analog  and  digital  electronics  techniques  and 
hardware  needed  to  achieve  the  60  Hz  frame  rate  operation,  good  resolution  uniformity, 
color  convergence  at  high  resolution  and  target  insetting  capability  required  of  the 
projector,  and  also  defined  a  cost  and  platform -weight  optimized  implementation  of  a 
specification-compliant  projector.  It  was  concluded  that  such  a  projector  is  feasible, 
but  only  if  the  CRT,  light  valve  and  prism  could  be  scaled  up  from  2  to  3  inches,  and  if 
the  resolution  of  the  CRT  could  be  improved. 

In  the  second  (component  development)  phase  of  the  program,  Hughes  developed  these 
critical  components.  A  3  inch  0. 7  mil  fiber  optic  CRT  was  built;  however  the  fiber 
optic  quality  of  the  fiber  optic  faceplate  was  not  acceptable,  and  the  second  CRT  was 
built  in  the  2  inch  size  to  work  around  the  unavailability  of  3  inch  high  quality  CRT  face¬ 
plates.  The  Unewldths  of  this  second  CRT — 0.  5  mils  and  0. 9  mils  for  the  target  and 
background  rasters,  respectively—  were  significantly  betther  than  the  required  0. 6 
mil  and  1. 1  mils,  respectively.  An  index-oil  prism  technique,  which  Immerses  the 
thin  film  polarizer/analyzer  beamsplitter  plate  in  an  index-matched  oil,  yields — 
when  used  in  conjunction  with  a  similarly  constructed  prepolarizer  prism — 
a  prism  free  of  residual,  stress  and  thermal  birefringence  effects.  1000:1  (a  factor  of 
five  higher  than  required).  A  partially  completed  effort  proved  the  basic  .  asibility 
of  increasing  the  size  of  the  CdSLV  from  2  to  3  inches.  A  breadboard  projector  constructed 
during  this  phase  to  integrate  these  components  under  conditions  equivalent  to  those 
encountered  in  the  ultimate  2363  projector  demonstrated — using  the  2  inch  high  resolution 
CRT  and  different  light  valves — that  an  MTF  of  29%  and  10%  is  achievable  at  2*  of  arc 
resolution  (across  a  90°  FOV),  using  a  high  resolution  CdS  light  valve  and  a  good  silicon 
light  valve,  respectively. 
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As  a  result  of  a  concurrently  funded  AFHRL  research  contract,  Hughes  internal  R&D, 
and  funding  redirected  within  the  2363  project,  major  strides  have  been  made  in 
implementing  a  practical,  operational  silicon  photoconductor  light  valve.  From  late 
1979  through  1980,  2  inch  fiber  optic  light  valves  were  constructed  and  demonstrated 
in  two  types  of  operating  projectors,  driven  by  fiber  optic  CRTs  to  display  dynamic 
television  images.  Liquid  crystal  response  time  was  improved  to  permit  operation 
at  60  Hz  frame  rates.  Device  construction  was  simplified  by  eliminating  a  separate 
light  blocking  layer.  Silicon  and  mirror  processing  techniques  were  improved  to  yield 
better  sensitivity  and  reduced  operating  voltages.  High  contrast  was  achieved  at  low 
brightness  levels,  and  limiting  resolution  was  improved  to  70  TV  lines/mm.  Two 
improvements  are  required  to  make  the  device  practical  for  simulation  applications: 
improving  the  light  blocking  characteristics  of  the  mirror  to  yield  higher  light  output 
with  good  contrast  ratio,  and  developing  a  technique  for  attaching  the  flat-polished 
wafer  to  the  fiber  optic  substrate  to  yield  the  required  brightness  uniformity.  Two 
further  SiLV  improvements  are  needed  to  meet  the  requirements  of  the  2363  projector: 
increasing  the  light  valve  size  by  a  factor  of  1.67  to  3  inches,  and  improving  its  resolu¬ 
tion  to  equal  that  of  a  high  resolution  CdSLV.  Except  for  the  last  item,  all  four  of  these 
improvements  entail  "device  engineering"— as  opposed  to  device  R&D— and  are  expected 
to  be  achievable  with  a  diligent  effort.  Further  improvement  of  resolution  is  also 
expected  when  the  inherent  resolution  capability  of  the  100  grid  line/mm  structure  is 
more  completely  utilized,  though  with  less  certainty. 
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INTRODUCTION 


The  Advanced  Simulator  for  Undergraduate  Pilot  Training  (ASUPT)  was  a  project 
derived  from  a  DOD  directive  to  the  three  services  requesting  programs  of 
advanced  development  in  the  area  of  training  and  education.  The  purpose  was 
to  insure  that  military  training  and  education  make  the  fullest  use  of  recent 
technological  advances.  The  ASUPT  was  a  research  device  designed  for  investi¬ 
gating  the  role  of  simulation  in  the  future  Undergraduate  Pilot  Training  (UPT) 
program.  The  development  of  the  ASUPT  system  began  in  1971  and  was  com¬ 
pleted  in  1975.  In  1977  the  A-10  cockpit  was  integrated  into  the  ASUPT 
simulator.  The  usefulness  of  ASUPT,  as  a  more  general  purpose  research  device, 
was  recognized  and  the  name  was  changed  to  Advanced  Simulator  for  Pilot  Train¬ 
ing  (ASPT).  Since  then  a  F-16  cockpit  configuration  has  also  been  integrated 
into  the  system. 

The  visual  system  of  the  ASPT  is  an  infinity  image  system  that  utilizes  7 
facets  of  a  dodecahedron  structure  to  provide  a  wide  field  of  view  display. 
Each  facet  contains  a  Farrand  Pancake  Window® optical  system  and  utilizes  a 
large  36"  diameter  CRT  as  a  display  source. 


In  1979,  General  Electric  was  contracted  to  develope  an  alternative  functional 
replacement  for  the  36"  ASPT  CRT  which  made  use  of  a  standard  production  GE 
monochrome  Light  Valve  TV  Projector.  The  goals  of  this  alternative  display 
development  are: 

1.  Investigate  the  use  of  the  Light  Valve  TV  Projector  technology  (with 
regard  to  resolution,  light  output  capabilities,  reliability,  maintainability, 
and  utility)  as  a  feasible  display  imagery  source  in  flight  simulators  and 
with  flight  simulator  motion  platforms. 

2.  Investigate  the  specific  application  and  use  of  a  lower  cost,  com¬ 
mercially  available  display  device  in  the  flight  simulator  and,  specifically, 
in  the  ASPT  system. 

3.  Develope  an  alternative  image  display  source  for  the  ASPT  System  which 
uces  a  36"  CRT  that  was  specially  designed  and  developed  for  the  ASPT  System. 

This  paper  will  describe  this  alternative  display  development.  In  addition,  a 
rent  program  to  develope  a  color  alternative  display  using  a  dual  GE  color  light 
valve  based  system,  which  provides  a  slewable  High  Resolution  Area  (HRA)  within 
the  background  (BG)  scene,  will  be  described. 
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THE  ALTERNATE  DISPLAY  SYSTEM  DESCRIPTION 

The  concept  of  the  alternate  display  system  is  to  replace  the  large  36"  CRT 
phosphor  surface  with  a  spherical  rear  projection  screen.  Imagery  is  project¬ 
ed  upon  this  screen  such  that  the  observer  views  the  CGI  scenery  through  the 
Pancake  Window*" optical  system. 

The  CIG  scenery  is  generated  by  projecting  upon  a  plane  normal  to  the  line  of 
sight  from  the  computed  eye  reference  point  position  to  points  in  the  data 
base  within  the  field  of  view.  This  two  dimensional  representation  of  the 
scene  is  converted  into  a  video  signal  and  fed  to  the  display  source.  This 
view  plane  scene  computation  concept  is  shown  in  Figure  1.  The  scene  must 
be  displayed  to  the  observer  as  though  he  were  looking  directly  at  this  view 
plane  to  preserve  the  scene  perspective. 

The  display  source  and  optical  system  must  have  the  appropriate  mappings  to 
prevent  scene  perspective  changes  and  distortions.  The  Pancake  Window® optical 
system  may  be  functionally  described  as  a  monocentric  optic  that  gives  true 
angle  imagery.  That  is,  for  an  object  point  on  the  CRT  or  projection  screen, 
the  observed  angle  at  the  eye  reference  point  is  the  same  as  the  angle  from 
the  center  of  curvature  of  the  CRT.  This  is  shown  in  Figure  2. 

The  alternate  display  system  configuration  is  shown  in  Figure  3.  The  alter¬ 
nate  display  system  components  are  the  light  valve  projector,  projection  lens, 
rear  projection  screen,  and  the  mechanical  mountings. 

The  display  source  is  a  General  Electric  model  PJ  7155  monochrome  light  valve, 
which  accepts  electrical  video  signals.  Properties  of  the  light  valve  are 
given  in  Table  1.  The  light  valve  utilizes  the  video  signal  to  velocity 
modulate  an  electron  beam  scanning  in  the  horizontal  direction,  which  results 
in  a  diffraction  grating  being  formed  on  a  transparent  fluid  surface.  Light 
passing  .through  an  input  bar  plane,  the  fluid  surface,  and  a  Schlieren 
optical  system  produces  light  representation  of  the  electrical  video  signal 
on  the  screen.  The  light  source  is  an  Xenon  lamp.  The  projector  is  comDlete- 
ly  self-contained  with  all  circuitry  and  power  conditioning  integrated  into 
a  single  unit. 

The  projection  lens  projects  the  light  valve  raster  upon  the  rear  projection 
screen.  Due  to  the  mapping  of  the  Pancake  Window®,  the  projection  lens  map¬ 
ping  must  be  defined  so  that  the  object  height  at  the  raster  p^ane  is  related 
to  the  angle  0  at  the  rear  projection  screen  by  the  relation, 

Y  *  f  tan  (ft) 

Where,  Y  =  raster  object  height 

f  *  effective  focal  length  of  the  projection  lens 
O-  *  the  angle  made  by  the  screen  image  height  as 
measured  from  the  screen  center 

A  lens  with  an  f  tan  (0)  mapping  and  no  exit  pupil  motion  versus  field  of  view 
would  meet  this  requirement  if  the  pupil  were  placed  at  the  projection  screen 
center. 

The  projection  lens  of  the  alternate  display  is  a  folded  22  element  design 
that  includes  a  Pechan  prism  derotation  device.  The  lens  design  was  derived 
from  previously  designed  wide  field  of  view  projection  lenses  designed  for  large 
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dome  real  image  display  applications.  Optical  performance  parameters  of  the  lens 
design  are  given  in  Table  2.  The  90°  circular  field  of  view,  of  the  lens, 
provides  imagery  for  the  so  called  overscan  region  of  the  display,  that  is,  it 
permits  the  required  6"  radius  sphere  of  head  motion  from  the  reference  eye  point. 
The  lens  is  designed  to  optically  and  mechanically  interface  with  the  light 
valve  Schlieren  objective.  Translational  and  angular  adjustments  and  a  means  for 
alignment  were  designed  into  the  mechanical  package  to  aid  in  this  interface. 

The  lens  and  light  valve  package  are  shown  in  the  photograph  of  Figure  4. 


The  rear  projection  screen  is  an  acrylic  dome  with  the  same  radius  as  the  CRT 
phosphor  surface.  This  surface  has  been  ground  with  a  loose  abrasive  technique 
to  achieve  a  gain  of  8.0.  A  light  shroud  between  the  lens  and  the  screen 
minimizes  potential  stray  light  problesm. 

Due  to  the  requirements  of  the  Air  Force  to  place  the  display  in  any  channel 
of  the  ASPT  dodecahedron,  a  versatile  mechanical  mounting  configuration  was 
designed.  The  light  valve  axis  must  be  positioned  horizontal  within  15°, 
although  short  term  large  angular  excursions  do  not  affect  performance.  The 
mechanical  mounting  is  designed  such  that  the  light  valve  can  be  rotated 
about  its  axis  and  the  light  valve/lens  assembly  can  be  rotated  to  discrete 
positions  about  the  axis  of  the  Pancake  Window^*.  Utilizing  rotations  about 
these  two  orthogonal  axes,  the  light  valve  axis  can  be  positioned  such  that 
for  the  mean  motion  playform  position  the  light  valve  axis  is  horizontal. 

The  image  rotation  resulting  from  the  particular  installation  facet  is  remov¬ 
ed  by  rotation  of  an  optical  derotation. 

PERFORMANCE  OF  THE  ALTERNATE  DISPLAY 


The  alternate  display  evaluation,  although  not  complete,  at  the  writing  of 
this  paper,  is  underway  and  qualitative  comments  on  the  display  system  are 
very  positive.  The  reduced  raster  promi nance  appears  to  more  than  compensate 
for  the  slightly  reduced  resolution  of  the  light  valve  versus  CRT  imagery. 
Alignments,  edge  matching  and  installation  procedure  of  the  display  have 
received  very  favorable  response  as  well. 

Serveral  areas  of  improvement  to  the  current  alternate  display  have  been 
identified.  These  areas  are: 

1)  the  rear  projection  screen 

2)  Color  matching  to  the  CRT  channels 

3)  minor  mechanical  modifications 

The  gain  of  the  current  rear  projection  screen  is  approximately  8.0.  This 
achieved  a  600  foot  lambert  screen  luminance,  however,  the  gain  affected  the 
luminance  of  the  display  versus  head  motion.  A  new  screen  candidate  fabricat¬ 
ed  by  using  a  flowing  of  a  dispersive  fluid  appears  very  promising.  It  offers 
a  much  higher  resolution,  a  more  uniform  appearance  and  a  gain  of  5.0  to 
improve  the  luminance  versus  head  motion  problem.  The  black  and  white  light 
valve  imagery  does  not  match  the  black  and  green  CRT  imagery.  A  dichroic 
filter  may  be  added  to  the  light  valve  to  match  the  light  valve  imagery  to 
the  CRT  phosphor  imagery.  An  added  benefit  of  this  modification  is  that  it 
would  permit  an  increase  in  the  Xenon  lamp  output  that  would  more  than  account 
for  the  lost  luminance  in  going  from  a  screen  gain  of  8.0  to  5.0.  Several 
mechanical  modifications  could  be  made  to  make  the  alternate  display  even 
easier  to  align  and  maintain. 
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THE  DUAL  PROJECTOR  DISPLAY 


The  resolution  versus  field  of  view  trade-off  in  simulation  has  been  approach¬ 
ed  in  several  display  systems  by  adding  a  high  resolution  scene  to  a  back¬ 
ground  scene.  The  dual  projector  display  concept  is  to  optically  combine 
a  90°  background  scene  with  a  servo  slewable  high  resolution  scene.  The 
display  system  concept  is  shown  in  Figure  5.  The  system  uses  two  GE  model 
5150  color  light  valves  as  sources.  The  background  light  valve  imagery  is 
projected  via  the  background  projection  lens  upon  the  rear  projection  screen 
to  provide  a  90°  field  of  view  to  the  observer.  This  projection  lens  path 
leads  through  a  hole  in  a  slewable  mirror.  Reflected  from  this  mirror  is  the 
high  resolution  scene,  which  can  be  slewed  to  any  point  within  the  background 
scene  or  can  be  entirely  outside  the  background  scene. 


Goals  for  the  Dual  Projector  Display  as  a  research  vehicle  require  much 
flexability  of  the  system.  The  goals  of  the  high  resolution  scene  perfor¬ 
mances  are  given  in  Table  3. 

The  Optical  Servo  Subsystem  (OSS)  portion  of  the  Dual  Projector  Display  is 
currently  in  the  late  stages  of  design.  The  formidable  optical  design 
task  is  virtually  complete  and  predicted  performance  for  the  system  is 
close  to  meeting  the  goals  of  the  program.  Delivery  of  the  OSS  is  expected 
by  July  31 ,  1981 . 

This  project  was  sponsored  by  the  PMTRADE  engineering  R  &  D  group  under  the 
direction  of  Dr.  Ron  Hofer  as  a  joint  Army/Air  Force  project.  The  Army  has 
provided  funding  and  GFE  hardware  for  this  effort  and  will  participate  in  the 
planned  testing  and  evaluation  in  October  of  1981. 

FUTURE  LIGHT  VALVE  DISPLAYS 

In  addition  to  the  GE  light  valve,  there  are  several  other  light  valves  in 
various  stages  of  development.  Of  the  currently  available  configurations, 
the  GE  system  appears  to  offer  the  best  luminance  to  size/weight  trade-off. 

The  systems  under  development  will  offer  potential  improvements  in  resolution. 
Future  improvements  of  the  GE  light  valve  are  also  being  investigated. 


SUMMARY 


An  alternate  display  system  to  replace  the  ASPT  36"  CRT  was  described. 

This  display  system  offers  monochrome  (black  and  White)  imagery  with  the 
ability  to  be  modified  for  color  imagery.  The  imagery  has  a  greatly  reduced 
raster  prominence  and  slightly  reduced  resolution.  In  addition,  the  maintenance 
of  the  alternate  display  offers  advantages  over  the  CRT  display.  The  display  is  a 
viable  option  for  the  presently  used  CRT. 

A  current  development  program  for  a  color  dual  projector  display  was  also  briefly 
described.  The  system  expected  to  be  demonstrated  in  October  1981  is  versatile 
in  providing  a  high  resolution  area  which  is  slewable  within  the  background. 
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TABLE  1 

GENERAL  ELECTRIC  LIGHT  VALVE  PROJECTOR  SPECIFICATIONS 


PJ  7155  (Monochrome)  PJ  5150  (Color) 


Output  (lumens) 

Typical 

Minimum 

Limiting  Resolution 
(TV  lines  per  picture  height) 
Horizontal 
Vertical 

Input  Power 
Watts 

Volt-amperes 

Scan  Standard 
Lines 

Fields  per  second 

Size 


2400 

650 

2000 

500 

800 

750 

750 

650 

1500 

1600 

2000 

2150 

1023 

1023 

60 

60 

22"  high,  17"  wide,  and  32"  long 


ALTERNATE  DISPLAY, 

FIELD  OF  VIEW 
F/NUMBER 

OPTICAL  DISTORTION 

DEROTATION  DEVICE 
IMAGE  RADIUS 
RELATIVE  ILLUMINATION 
MISCELLANEOUS 


TABLE  2 

PROJECTION  LENS  DESCRIPTION 


90°  TOTAL 


3.0 


2.9%  MAXIMUM 


PECHAN  PRISM 


609.6 


0.85 


1  INTERMEDIATE  IMAGE 
FILTER  CAPABILITY 
1  FOLD  MIRROR  FOR  USE  IN  ANY 
ASPT  DISPLAY  CHANNEL 
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TABLE  3 

DUAL  PROJECTOR  DISPLAY  SYSTEM  GOALS 

1.  Provide  a  1.0  arc  minute  average  resolution  scene  that  is  slewable 
to  any  point  in  the  lower  resolution  background  scene. 

2.  High  resolution  slew  rates  of  300°  per  second  that  can  be  head  or 
target  tracked. 


3.  High  Resolution  display  modes: 
inset,  abrupt  transition 
inset,  gradual  transition 
superimposed 
circular  field  of  view 
square  field  of  view 


4.  Develop  electronic  insetting  techniques. 


5.  Provide  a  modular  design  that  is  optically  and  mechanically  interchangeable 
with  any  ASPT  channel. 


EYE  REFERENCE  POINT 


SCENE  COMPUTATION  GEOMETRY 
FIGURE  1 


PANCAKE  WINDOW 


PANCAKE  WINDOW  MAPPING  CHARACTERISTIC 
FIGURE  2 


dodecahedron  structure 


DISPLAY  ADJUSTMENT  PLATFORM 


THE  ALTERNATE  DISPLAY  SYSTEM 
FIGURE  4 
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DODECAHEDRON  STRUCTURE 


DUAL  PROJECTOR  DISPLAY  CONCEPT 
FIGURE  5 
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Francois  Desvignes 

Director 

SODERN 


Jean  R.  Huriet 

TITUS  program  manager 

SODERN 


■I 


Frangois  Desvignes  was  educated  in  physics 
at  the  CNAM  Paris,  1947  with  a  specializa¬ 
tion  in  optics  (ESO  Paris,  1945).  His  35 
years  of  professional  life  have  been  in 
Research,  Industry  and  academia  as  Head  of 
the  Photometry  Laboratory  at  the  Institut 
d'Optique  (Paris)  since  1946  ;  as  Department 
Head  of  the  Laboratoires  d ' Electronique  et 
Physique  appliquee  in  1951  with  activities 
on  radiometry,  solid  state  photoelectricity 
and  infrared  instrumentation  ;  at  SODERN 
since  1971  where  he  is  presently  Director, 
to  develop  an  electro-optical  instrumentation 
activity  (satellite  attitude  sensors.  Earth 
resources,  cameras,  ...)  which  includes  the 
video-image  projection  systems  and  as  a 
Professor  at  the  Ecole  Superieure  d'Optique 
in  the  fields  of  photometry,  radiometry, 
radiation  detection  and  detectors  (1962-78) . 


Jean  R.  Huriet  obtained  his  engineering 
diploma  at  the  Institute  for  Optics  in 
Paris  and  then  he  became  doctor-engineer 
at  the  Paris  University.  His  20  years 
professional  experience  has  been  in  re¬ 
search  laboratory,  satellite  experiment 
management,  before  he  came  to  Sodern  in 
1970  for  the  space  projects  management. 
He  is  in  charge  of  the  TITUS  projection 
system  program  since  1978. 


233 


TITUS  LIGHT  VALVE  PROJECTION  SYSTEM 
(TLVPS) 

by  F.  DESVIGNES  and  J.R.  HURIET  -  SODERN  -  LIMEIL  -  France 


ABSTRACT  : 


The  TLVPS  has  been  developped  for  large  screen  TV  display 
in  1976.  Additional  development  work  is  running  in  the  frame  of  the 
USAF  2363  program  and  for  the  Mirage  2000  simulation.  The  goal  is 
to  obtain  high  resolution  areas  (2  arcs  minutes)  inserted  in  a 
medium  resolution  (6  arcs  minutes)  background. 

oOoOoOoOoOoOo 


Five  years  ago,  SODERN  started  the  manufacturing  of  full 
colour  projectors  adapted  to  European  live  television  standards, 
which  provide  an  output  greater  than  2500  lumens,  convenient  for 
movie  theatre  screens  up  to  100  square  meters  (900  square  feet) . 
These  laboratory  models  have  been  intensively  used  for  demonstra¬ 
tion  purpose  since  1977  and  thus  proven  their  reliability  and  low 
level  of  maintenance  requirements. 

The  images  produced  by  this  type  of  projector  have  uni¬ 
que  features  in  addition  to  the  high  luminous  output  : 

-  full  colour  capability  (see  figure  1), 

-  neither  flicker  nor  lag, 

-  memory  capability  without  smear, 

-  no  apparent  line  structure  with  raster  scan  mode. 

The  resolution  obtained  with  this  first  experimental 
projection  system  is  considered  of  not  sufficient  for  the  military 
aircraft  -  air  combat  and  ground  attack  -  simulation  needs  ; 
improvement  programmes  have  been  undertaken  with  the  US  Air  Force 
and  the  French  Delegation  Generale  pour  l'Armement  support*. 

The  performance  can  be  expressed  in  terms  of  space  and 
time  resolution  under  the  pixels  rate  figure  of  merit  (number  of 
resolved  picture  elements  per  unit  of  time) . 


*  2363  USAF  project  and  Mirage  2000  simulation. 
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This  projector  figure  of  merit  takes  into  account  three 

terms  : 

1  -  the  light  valve , 

2  -  the  video  electronics, 

3  -  the  optics. 

Presently  the  last  one  is  not  critical  from  the  resolu¬ 
tion  point  r  f  view  :  the  development  effort  for  the  resolution  is 
put  on  the  light  valve  and  on  the  electronics. 

The  light  valve  uses  the  longitudinal  Pockets  effect,  an 
electrically  induced  optical  birefringence  which  is  produced  here 
in  a  thin  slice  of  a  synthetic  crystal.  An  electrical  image  is 
written  on  the  slice  with  a  scanning  electron  beam.  The  light  valve 
spatial  resolution  is  mostly  related  to  three  parameters  :  the 
erasing-writing  electron  beam  diameter,  the  crystal  slice  lateral 
dimensions,  the  spacing  between  the  slice  and  a  control  grid  which 
limits  secondary  electrons  fallout. 

The  erasing-writing  speed  is  related  to  the  electron  beam 
current,  thus  the  pixels  rate  is  controlled  on  the  spatial  and  time 
resolution  aspects  by  the  electron  beam  characteristics. 

The  1980  effort  has  been  concentrated  on  the  electron  gun 
improvement  which  has  produced  an  increase  of  the  light  valve  reso¬ 
lution  as  shown  by  figure  2.  A  30  %  contrast  transmittance  (MTF)  is 
obtained  at  550  pixels  per  line  with  the  new  valve  (TV2)  against 
5  %  with  the  1977  one  (TV1),  and  a  5  %  MTF  is  obtained  at  100  pixels 
per  line. 


A  new  improvement  of  the  electron  gun  is  still  possible. 

An  other  way  of  investigation  to  increase  the  available 
number  of  pixels  per  image  is  to  increase  the  size  of  the  target 
of  the  light  valve.  The  TV1  and  TV2  light  valves  use  a  28x38mm 
target.  A  SV1  light  valve  has  been  made  and  tested  with  a  38x38mm 
target. 


The  figure  3  is  a  picture  of  the  display  of  a  test  pattern 
with  the  SV1  light  valve.  The  observed  MTF  is  the  same  as  for  the 
TV2 ,  then  the  benefit  in  available  pixels  number  is  directly  pro¬ 
portional  to  the  area  increasing  :*m-  30  %.  It  can  be  used  in  two 
different  ways  either  to  fit  with  square  shaped  or  pentagonal  ima¬ 
ges  or  to  increase  the  effective  resolution  in  a  rectangular  image 
by  use  of  anamorphosal  optics. 
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During  the  same  time  investigations  have  been  made  in 
order  to  grow  larger  crystals  where  the  targets  are  cut.  The  DKDP 
crystals  which  allow  to  make  the  38x38mm  targets  have  a  50x50mm 
size.  Bigger  crystals  have  been  grown  up  to  70x70mm  which  allow 
to  cut  50x50mm  targets  (see  figure  4) .  The  electro-optical  quality 
have  been  checked  and  sample  targets  made  to  check  also  the  image 
quality  after  complete  processing. 

By  the  other  hand  a  development  work  has  been  undertaken 
on  the  system  aspect  of  the  projector.  That  leads  mostly  to  the 
improvement  of  the  command  circuitry  of  the  light  valves.  The 
scanning  capabilities  will  be  extended  to  1023  lines  to  take  be¬ 
nefit  of  higher  resolution  and  the  video  amplifier  (140  V  on 
400  pF  load)  bandwith  is  progressively  extended  from  5  to  7,  15 
and  possibly  30  MHz. 

But  it  leads  also  to  the  repackaging  of  the  projector 
in  order  to  obtain  suitable  si2e  and  weight  for  simulation  purpose, 
a  prototype  demonstrating  the  1980  improvement  will  be  available 
this  year. 
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XY  COLOR  DIAGRAM 


RVB  TITUS  protection  system 
R'V'B'  Typical  shadow  mask  tube 
R*V"B"  NTSC  reference  diagram 


IMPROVEMENT  OF  THE  LIGHT  VALVES 
TV2  VERSUS  TV1  CHARACTERISTICS 
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ABSTRACT 


A  display  system  for  flight  simulation  is  described.  The  system  employs 
optics  mounted  on  the  pilot's  helmet  to  project  a  scene  onto  a 
retro-reflecting  screen.  It  is  driven  by  two  Computer  Image  Generation  (CIG) 
channels,  one  providing  a  wide-angle  display  while  the  other  provides  a 
high-resolution,  eye-directed  inset.  The  concept  uses  both  head  tracking  and 
eye  tracking  to  provide  an  effectively  unlimited  field  of  view  with  high 
resolution  at  low  cost. 

1.  INTRODUCTION 


The  Advanced  Simulation  Concepts  Laboratory  of  the  Naval  Training 
Equipment  Center  is  pursuing  an  exploratory  development  program  to  design  a 
visual  simulation  system  and  analyze  the  feasibility  of  fabrication.  The 
system  will  be  designed  for  use  in  the  Navy  Visual  Technology  Research 
Simulator  (VTRS)  with  new  hardware  intended  to  interface  with  existing  VTRS 
equipment . 

The  system  will  have  part  of  the  display  projection  optics  mounted  on  the 
pilot's  helmet  and  be  fully  visually  coupled,  demanding  use  of  both 
head-tracking  and  eye-tracking  techniques.  Optical  design  for  the  display 
system  is  currently  well  advanced  and  experiments  have  been  carried  out  in 
both  head-  and  eye-tracking.  This  paper  describes  and  discusses  the  evolution 
of  the  basic  approach  taken,  and  outlines  the  present  design  and  the  results 
of  experiments. 

The  system  performance  objectives  include  an  effective  total  field  of 
view  which  is  limited  only  by  cockpit  structure  and  the  pilot's  normal  freedom 
of  movement  within  the  cockpit.  Associated  with  this  large  field  of  view, 
typically  8  steradians,  the  target  resolution  is  3.3  arc  minutes  per  line 
pair.  The  scene  is  to  be  presented  without  noticeable  seams  or 
discontinuities,  and  is  to  be  available,  without  distortion,  to  two  or  more 
subjects  in  the  same  simulated  aircraft.  The  method  proposed  has  potential 
for  presenting  a  collimated  picture  and  eventually  a  three-dimensional 
picture. 

2.  DESIGN  CONCEPTS 


a.  Visual  Coupling 

The  very  demanding  specification  is  achievable  in  principle  by  many 
different  approaches.  But  generally,  the  cost  of  attaining  the  basic 
objectives  -  very  wide  field,  high  resolution  and  multiple-user  operation  - 
would  be  considered  excessive  using  available  display  technology.  In  general, 
the  wide  angle  and  high  resolution  objectives  together  demand  a  powerful  pixel 
generation  capability,  leading  to  multiple  projectors  and  very  broad 
electronic  bandwidths,  and  to  multiplied  cost  of  computer  image  generation  or 
other  video  generation  systems.  The  requirement  for  multiple-user  operation, 
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taken  together  with  the  very  large  field  angles,  would  In  general  require  very 
costly  collimation  optics,  or  else  duplication  of  the  entire  simulation 
apparatus  (with  large  actual  separation  of  the  users). 

The  approach  taken  is  based  on  matching  the  display  system 
performance,  in  terms  of  field  of  view  and  resolution,  to  the  visual 
performance  parameters  of  the  observer's  eyes.  The  observer's  capability  to 
perceive  high  detail  at  any  instant  is  restricted  to  a  relatively  small  area 
of  interest  corresponding  to  his  foveal  vision.  The  size  of  the  instantaneous 
field  in  which  he  can  perceive  any  visual  information  at  all  is  less  than  the 
field  available  to  the  observer  through  head  and  body  movements.  By  providing 
a  display  system  which  presents  high  detail  imagery  only  where  the  observer  is 
looking,  with  an  instantaneous  field  which  matches  his  instantaneous  field, 
the  observer  will  perceive  the  display  as  having  high  detail  imagery 
throughout  his  total  available  field.  But  the  total  burden  on  the  display 
system,  in  terms  of  computed  and  resolved  pixels  in  each  frame,  is  very 
usefully  limited.  For  these  reasons,  there  has  been  a  strong  thrust  in  recent 
years  to  visually  coupled  systems  of  different  kinds,  in  which  the  projected 
scene  or  a  high-detail  insert  follows  the  observer's  head-  or  eye-direction. 

b.  NTEC  System  Goals 

Figure  1  shows  an  artist's  concept  of  the  NTEC  display  looking  over 
the  observer's  shoulder.  The  display  has  an  eye  directed  area  of  interest 
(AOI)  field  which  is  a  nominal  1000  television  line  (TVL)  raster  driven  by  a 
dedicated  CIG  channel  and  a  head  directed  instantaneous  field  of  view  (1F0V) 
which  is  a  second  nominal  1000  TVL  raster  driven  by  a  second  CIG  channel.  The 
system  is  largely  compatible  with  the  existing  VTRS  computer  image  generator 
(Ref.  1).  Table  1  lists  the  system  performance  goals  chosen  for  the  design. 

The  VTRS  CIG  has  a  capability  of  providing  2,000  potentially 
visible  edges  distributed  between  the  two  display  rasters.  If  this  capacity 
is  shared  equally  between  the  displays,  the  scene  complexity  or  edge  density 
will  be  approximately  twenty  times  higher  in  the  AOI  channel  relative  to  the 
IF0V  channel. 
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TABLE  1.  SYSTEM  PERFORMANCE  GOALS 


Apparent  Field  of  View 

Instantaneous  Field  of  View 

Area  of  Interest 

Apparent  Resolution 

AOI  Resolution 

1FOV  Resolution 

Luminance 

Color 

Contrast 


Restricted  only  by  cockpit  structure 
145°  diagonal 
36°  diagonal 

3.3  arc  min/TV  line  pair 
3.3  arc  min/TV  line  pair  -  on  axis 
13  arc  min/TV  line  pair  -  on  axis 
10  Foot  Lamberts  (Highlight) 

Full 

30:1 


c.  Helmet  Mounted  Projector 

The  display  configuration  was  developed  from  a  Helmet-Mounted  Laser 
Projector  concept  described  in  a  contract  report  (Ref.  2)  delivered  by 
American  Airlines  and  Redifon  Simulation  Ltd.  Helmet-mounted  optical  systems 
commonly  have  small  CRTs  on  which  an  image  is  generated  and  the  optics  project 
light  into  one  or  both  of  the  user's  eyes  via  mirrors  or  beam  splitters 
located  in  front  of  the  user's  eyes.  In  the  Redifon  proposal,  the  scene  is 
projected  outward  from  the  helmet  onto  a  dome  screen  from  an  exit  pupil 
located  above  the  user's  eyes.  The  screen  is  given  a  strongly 
retro-reflective  coating  so  that  a  relatively  large  proportion  of  the 
projected  light  is  reflected  to  the  user.  In  order  to  achieve  high  brightness 
in  the  display  over  a  wide  angle,  without  mounting  excessively  heavy  optics  on 
the  helmet,  Redifon  proposed  to  use  a  laser  display  system  and  a  coherent 
fiber  optic  link  to  carry  light  to  the  helmet.  The  use  of  lasers  permits  high 
brightness  across  a  wide  projected  field  without  requiring  wide-aperture 
projection  optics.  The  flexible  fiber  link  permits  the  more  massive  optical 
components,  including  the  lasers,  modulators  and  line  scanner,  to  be  mounted 
at  a  remote  location,  off  the  helmet.  The  fiber  link  to  helmet  mounted  optics 
is  not  a  concept  unique  to  Redifon,  but  Redifon  has  made  the  fiber  link  itself 
more  viable  by  proposing  to  put  a  light-weight  frame  scanner  on  the  helmet. 
This  means  that  the  optical  fibers  are  required  to  transmit  line  images, 
rather  than  full  frame  images;  few  fibers  are  required,  with  significant 
advantages  in  flexibility  of  the  link  and  reduction  of  manufacturing 
difficulty. 


The  aspects  of  the  Redifon  concept  described  above  have  been 
retained  in  the  NTEC  development.  In  other  respects  the  concept  has  been 
modified,  mainly  to  reduce  the  likely  cost  and  time  required  to  build  a 
working  system. 
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Helmet  mounting  of  projection  optics  generally  has  several 
advantages  for  visually  coupled  displays: 

(1)  The  display  automatically  rotates  with  the  head-pointing 
direction. 

(2)  There  Is  no  significant  distortion  due  to  rotation  of  the 
projector. 

(3)  Good  pnotometric  efficiency  is  possible  (with  either  direct 
projection  Into  the  eyes  or  outward  projection  onto  a 
retro-reflecting  screen). 

(4)  Two  or  more  subjects  can  use  the  same  simulator  cockpit 
without  seeing  distorted  views  of  each  other's  head/eye 
directed  scenes  (cross-talk),  and  each  can  receive  an 
undistorted  scene  image. 

(5)  With  binocular  projection  the  technique  has  potential  for 
effective  collimation  of  the  scene,  and  for  3-dimensional 
presentation  (neither  at  present  planned  by  NTEC). 

d.  Dome  Screen 

Projection  outward  from  the  helmet  onto  a  screen,  rather  than 
projection  directly  into  the  pilot's  eyes,  was  selected  for  these  reasons: 

(1)  It  does  not  require  beam  splitters  or  any  other  hardware  to 
be  fixed  immediately  before  the  pilot's  eyes  thus  obstructing 
a  normal  view  of  the  cockpit  interior. 

(2)  It  permits  a  very  wide  instantaneous  field  of  view, 
comparable  with  the  pilot’s  total  field  with  head  fixed,  to 
be  projected  and  viewed  by  both  eyes. 

(3)  It  provides  automatic  and  precise  blanking  of  the  projected 
outside-world  scene  at  the  cockpit  outline,  since  the  cockpit 

•  structure  does  not  retro-ref lect ,  without  need  for  head 
position  tracking  or  electronic  raster  blanking. 

e.  Head- Tracking  Functions 

Since  the  projected  rasters  rotate  with  the  user’s  head,  the  view 
direction  used  to  compute  the  scene  must  be  updated  at  field  rate  to  include 
head  rotations.  A  tracking  device,  measuring  pitch,  roll  and  yaw  of  the 
helmet  with  respect  to  the  cockpit,  must  therefore  be  included  in  the  visual 
simulation  system. 

Errors  in  helmet  attitude  data  transmitted  to  the  CIG  produce 
errors  in  location  of  the  perceived  scene.  Inadequate  precision  can  result  in 
image  jitter,  low  accuracy  will  produce  image  swimming,  and  inadequate 
response  will  cause  the  image  to  lag  head  movements. 
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Image  lag  due  to  CIG  throughput  delay  can  be  compensated,  since  the 
error  Is  known  given  fast  head-tracker  response,  by  offsetting  the  projected 
image  rasters  using  optical  deflectors. 

f.  Eye  Direction  Following  Functions 

The  small  AOI  raster,  with  high  resolution  and  high  detail  content, 
is  required  to  be  shifted,  within  the  wide  field  of  the  helmet  mounted 
projector,  so  that  this  area  is  always  on  the  axis  of  the  pilot's  eyes. 

Within  the  optical  projection  hardware,  this  requires  provision  of 
deflectors  to  shift  the  scene  vertically  and  horizontally.  Within  the  CIG 
system,  it  is  necessary  to  provide  corresponding  shifts  in  computed  locations 
of  the  AOI  view  window  and  raster.  Data  to  determine  the  eye-following 
deflections  must  be  provided  by  an  eye  attitude  measuring  system. 

3.  DISPLAY  SYSTEM 


Figure  2  shows  a  functional  diagram  of  the  display  system. 

Mounted  off  the  helmet  are  lasers,  intensity  modulators  and  a  line 
scanner.  These  components  provide  two  intensity  modulated  line  scan  images, 
one  for  the  AOI  raster  and  one  for  the  IFOV.  The  two  line  scan  images  are 
carried  to  the  helmet-mounted  components  by  two  flexible  coherent  fiber  optic 
bundles. 

Mounted  on  top  of  the  helmet  are  galvanometers  driving  flat  mirrors. 
These  provide  frame  scan  and  eye-following  offsets  of  the  projected  rasters. 
The  frame-scan  and  offsetting  optics  are  followed  by  a  compound  projection 
lens  system  which  relays  the  light  to  an  exit  pupil  location  in  front  of  the 
user's  forehead  and  projects  the  light  outward  toward  the  display  screen 
providing  the  angular  magnification  required  to  fill  a  145°  Instantaneous 
field. 


The  screen  is  a  spherical  dome  surrounding  the  simulator  cockpit  giving 
the  user  a  total  field  (with  head  and  body  movements)  which  is  limited  only  by 
cockpit  structure.  The  screen  is  coated  with  a  retro-reflecting  material. 

The  CIG  system  has  inputs  from  head-tracking  and  eye-tracking  devices, 
not  indicated  in  Figure  2.  These  inputs  determine  rotations  of  the  view 
direction  to  follow  head  orientation,  and  computed  offsets  of  the  view  window 
and  raster  to  follow  eye  attitude.  In  addition  to  providing  video  signals  to 
modulators  and  controlling  simple  line  and  frame  scan  functions,  the  CIG 
system  controls  offsets  in  the  projected  rasters  produced  at  helmet-mounted 
deflectors.  These  offsets  are  principally  for  eye  following  but  also 
compensate  for  errors  due  to  finite  computational  throughput  time. 

4.  LASERS,  MODULATORS  AND  LINE  SCANNER 


Figure  3  is  a  functional  diagram  of  the  optical  system  feeding  the 
helmet-mounted  projector. 
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DISPLAY  OPTICAL  SYSTEM 
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The  beam  from  an  Argon  Ion  laser,  operated  in  all-lines  output  mode,  is 
split  into  its  separate  wavelength  components  by  a  dispersive  optical  system. 
Part  of  the  power  at  514.5  nm  is  split  off  to  provide  a  green  beam,  and  power 
in  a  selected  set  of  blue  wavelengths  is  split  off  to  provide  a  blue  beam. 
Blue-green  components,  plus  residual  green  power,  are  used  to  pump  a  Rhodamine 
6G  dye  laser,  which  is  tuned  to  output  red  light  at  wavelength  610  nm. 
Dispersive  optics  are  used  to  recombine  separate  wavelengths  assigned  to  the 
blue  primary  onto  a  single  beam,  and  similarly  to  recombine  the  separate 
wavelengths  assigned  to  pumping  the  dye.  The  Argon  laser  output  power 
required  to  achieve  a  10  f.l.  display  brightness  is  estimated,  based  on 
pessimistic  assumptions,  at  10W. 

Red,  green  and  blue  beams  are  each  split  in  two  for  AOI  and  IFOV  rasters, 
and  the  six  separate  beams  are  intensity  modulated  at  acousto-optic 
modulators.  These  beams  are  then  combined  to  give  two  full-color  beams,  which 
are  then  scanned  at  a  common  line  scanner  and  imaged,  separately,  onto  two 
coherent  fiber  optic  bundles. 

The  line  scanner  is  a  rotating  polygon  having  24  facets,  rotating  at 
76,725  rpm  to  give  a  line  rate  of  30,690  lines/second  to  both  modulated 
full-color  beams.  The  line  scan  images  will  be  10  mm  long  by  10  microns 
wide.  Each  image  will  fall  on  a  single  row  of  1000,  10  micron,  optical 
fibers.  (The  flbei  bundles  may,  for  manufacturing  convenience  and  to  allow 
some  selection,  have  many  rows  of  1000  fibers  each.)  The  maximum  expected 
laser  power  density  on  the  input  end  of  the  IFOV  fiber  bundle  is  20  W/mm^, 
which  is  within  a  factor  2  of  a  measured  damage  threshold.  Therefore, 
tolerance  to  incident  power  is  an  important  part  of  the  fiber  optic 
procurement  specification.  The  fiber  optic  bundles  will  be  approximately  9' 
long,  permitting  the  lasers,  etc.,  to  be  remote  from  the  simulator  cockpit. 

5.  HELMET -MOUNTED  OPTICS 


The  design  for  optics  mounted  on  the  user's  helmet  is  shown  in  Figures  4, 
and  5. 


The  helmet-mounted  optics  include  mirror  scanners  driven  by 
galvanometers,  and  the  final  wide-angle  projection  lens  which  directs  the 
light  outward  to  the  display  screen.  The  mirror  scanners  are  used  to  generate 
frame  scan  for  the  two  projected  rasters,  AOI  and  IFOV.  They  also  provide  a 
capability  for  controlled  offset  of  the  two  rasters  with  respect  to  the  axis 
of  the  helmet  mounted  projector  lens.  Light  from  the  two  line  scan  images, 
input  by  fiber  optics,  is  first  passed  through  a  scanner  subsystem  which 
permits  the  two  line  scans  to  be  shifted  along  their  own  lengths.  This 
produces  a  line-direction  shift  of  the  projected  rasters,  normally  a 
horizontal  shift,  which  is  used  primarily  for  following  horizontal  eye 
rotations.  After  passing  through  relay  lenses,  the  two  beams  are  reflected  at 
separate  scanner  mirrors  mounted  on  the  shaft  of  a  single  galvanometer  so  that 
the  line  offset  mechanism  shifts  the  two  rasters  together. 

In  each  beam,  the  light  path  Is  folded  at  a  narrow  strip  of  flat  mirror 
on  a  window  located  at  a  line  scan  image.  The  light  is  then  recollimated  by  a 
spherical  mirror  and  relayed  onto  the  frame  scanners,  passing  through  the 
window. 
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^  FIGURE  4 

HELMET  •  MOUNTED  OPTICS 
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Two  flat  mirrors,  driven  by  separate  galvanometers,  provide  frame  scan 
and  also  combine  the  two  beams  on  a  common  optical  axis.  The  frame  scanners 
provide  controlled  offset  of  the  rasters  in  the  frame  scan  direction,  which  is 
normally  the  vertical  direction.  The  AOI  beam  is  deflected  at  a  single 
galvanometer-driven  mirror,  designated  the  AOI  mirror,  which  produces  the 
cyclic  linear  ramp  scan  required  for  the  AOI  raster.  The  IFOV  beam  is  first 
deflected  at  a  small  galvanometer-driven  raiiror,  designated  the  IFOV  mirror, 
which  generates  cyclic  linear  ramp  scan  to  produce  3/4  of  the  frame  dimension 
for  the  IFOV  raster.  The  IFOV  beam  then  falls  on  the  AOI  mirror  which  adds 
frame  scan  to  complete  the  IFOV  raster. 

The  two  beams  are  effectively  combined  at  the  small  IFOV  mirror.  The 
IFOV  beam  is  reflected  from  it,  while  the  AOI  beam,  which  has  a  larger 
diameter,  passes  around  the  mirror  with  partial  obscuration  by  it. 

Controlled  offset  of  the  rasters  in  the  frame  direction  is  provided  by  an 
offset  only  at  the  AOI  mirror.  Since  both  beams  are  deflected  at  this  mirror, 
the  two  rasters  are  deflected  together.  The  offset  is  used  primarily  to 
follow  vertical  eye  rotations. 

The  frame-scan  and  raster  offsetting  optics  will  be  located  on  top  of  the 
pilot's  helmet.  The  raster  images  generated  at  this  point  will  have  moderate 
field  angles,  AOI  and  IFOV  line  lengths  being  respectively  11°  and  44°.  Light 
emerging  from  the  frame-scan  system  is  relayed  by  a  complex  lens  and  mirror 
system  to  an  exit  pupil  which  will  be  located  in  front  of  the  pilot's 
forehead,  approximately  30  mm  above  his  eye  level  and  70  mm  forward  of  his 
eyes.  The  system  will  provide  angular  magnification  and  direct  the  combined 
raster  images  outward  onto  the  display  screen. 

The  final  lenses  will  produce  substantial  pincushion  distortion, 
corrected  in  CIG,  which  is  useful  in  distributing  resolution  optimally  across 
the  projected  field. 

6.  COCKPIT  AND  DISPLAY  SCREEN 


The  display  screen  will  be  a  ten  foot  radius  dome,  already  available  in 
the  VTRS  facility,  coated  with  retro-reflecting  material.  The  separation  of 
the  user's  eyes  from  the  projector  exit  pupil  will  be  approximately  three 
inches  so  that  the  screen  surface  will  be  required  to  provide  a  reasonably 
uniform  spread  of  light  within  1.5°  of  the  retro-reflective  direction.  A 
photometric  gain  in  excess  of  1000  is  in  theory  possible,  given  this 
requirement  on  beam  spread,  but  the  target  figure  for  screen  gain  is  100, 
which  is  considered  likely  to  be  achieved. 

Commercially  available  retro-reflecting  screen  materials  have 
characteristics  similar  to  the  required  material,  but  none  is  close  enough, 
and  therefore  a  special  product  is  required.  This  development  is  currently 
underway  through  a  contract  with  the  University  of  Arizona  Optical  Sciences 
Center. 

The  method  of  projection,  from  a  point  a  few  inches  from  the  user's  eye- 
point,  produces  potential  problems  due  to  light  falling  on  the  cockpit 
structure.  Internal  surfaces  of  the  cockpit  will  reflect  low-brightness 
images  of  the  projected  scene  and  it  is  necessary  to  ensure  that  these  Images 


253 


will  not  be  noticeable.  This  is  done  essentially  by  arranging  that  the 
photometric  gain  of  the  screen  is  very  much  higher  than  the  gain  of  cockpit 
surfaces  -  in  fact  by  three  to  four  orders  of  magnitude  -  so  that  the  relative 
brightness  of  the  screen  image  is  two  to  three  orders  higher  than  that  of 
ghosts  within  the  cockpit.  The  dim  ghost  images  on  the  cockpit  structure  will 
normally  not  be  visible  against  a  background  of  ordinary  internal  cockpit 
1 ight ing . 

The  wide-angle  scene  projected  onto  the  screen  will  have  sharp  boundaries 
due  to  shadowing  by  cockpit  structures.  Shadows  produced  by  the  lower  edges 
of  cockpit  windows  are  not  visible  to  the  user,  but  shadows  produced  by  window 
struts  have  been  found,  in  simple  experiments,  to  be  very  noticeable.  The 
helmet-mounted  projection  technique  is  likely  to  be  used  only  in  simulator 
cockpits  in  which  window  struts  are  omitted. 

Windows  themselves  will  also  be  omitted  to  avoid  possible  specular 
reflection  of  projected  light  by  the  windows  directly  to  the  user,  and  care 
will  be  taken  to  avoid  specular  reflections  to  the  pilot  from  cockpit 
instrument  surfaces. 

Use  of  the  helmet  mounted  display  will  require  some  relatively  minor 
alterations  to  the  simulator  heads-up  display.  The  heads-up  display  beam 
combiner  will  be  made  as  thin  as  possible,  and  it's  reflection  coefficient 
reduced,  so  that  it  will  not  produce  an  objectionable  displacement  or  shadow 
in  the  forward  area  of  the  projected  scene.  The  brightness  of  display  light 
reflected  to  the  pilot  from  the  heads-up  display  lens  and  CRT  will  be  low 
compared  with  the  brightness  of  the  screen  image.  The  CRT  location  will  be 
shifted  so  that  it's  virtual  image  falls  at  the  screen  distance. 

7.  SOME  DEVELOPMENT  PROBLEMS 

In  general,  the  approach  taken  in  design  of  the  display  system  has  been 
chosen  to  reduce  procurement  problems  to  a  minimum  so  that  a  visually  coupled 
system,  using  helmet  mounting,  can  be  assessed  as  soon  as  possible  and  at 
relatively  low  cost. 

Given  the  basic  idea  proposed  by  American  Airlines  and  Redifon, 
potentially  severe  development  problem  areas  include:  the  fiber  optic  link, 
the  helmet-mounted  frame  scanner  and  the  retro-reflective  screen.  The 
specification  on  the  fiber  optics  is  relaxed  considerably  by  allowing  the 
helmet-mounted  system  to  be  relatively  complex.  In  particular,  the  length  of 
the  AOI  line  image  to  be  carried  by  the  fibers  is  minimized  by  locating  the 
line  scan  offsetting  system  on  the  helmet;  and  the  output  end  configuration  of 
the  fiber  bundles  is  simplified  by  use  of  a  relatively  complex  flat-field  lens 
system  on  the  helmet.  The  complex  lens  system,  by  providing  angular 
magnification,  also  reduces  the  specification  on  frame  scanners  so  that 
existing  galvanometers  can  be  used.  The  complexity  of  the  helmet-mounted 
projection  system  which  is  evolved  on  these  lines  might  make  a  binocular 
system,  with  separate  optics  for  each  of  the  user's  eyes,  prohibitively 
bulky.  Therefore,  a  decision  was  made  not  to  attempt  a  binocular  arrangement, 
and  to  forego  effective  binocular  collimatlon  and  immediate  potential  for 
three-dimensional  projection.  This  greatly  eases  the  specification  on  the 
display  screen,  since  there  is  no  immediate  requirement  to  provide 
retro-reflection  selectively  to  each  of  the  user's  eyes. 
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Some  potentially  severe  electro-optics  problems  are  avoided  by  electing 
to  design  the  scanning  system  to  project  two  rasters,  although  this  requires 
two  separate  galvanometers  on  the  helmet  and  separate  relay  optics  up  to  the 
frame  scanners.  The  alternative  approach  is  generation  of  a  single  raster 
which  must  fill  the  IFOV  field  and  be  capable  of  the  resolution  required  for 
the  AOI.  This  implies  a  need  to  generate  resolved  pixels  at  rates  in  excess 
of  200  MHz.  Components  at  or  beyond  current  state-of-the-art  are  required 

(a)  to  provide  pixel  speed-up  for  the  CIG  signal  assigned  to  the  AOI  channel, 

(b)  to  intensity  modulate,  and  (c)  to  provide  line  scan.  The  fiber  optic  link 
for  the  single  raster  must  have  approximately  4000  fibers  in  the  row  used 
compared  with  1000  per  bundle  given  the  present  NTEC  approach.  If  two 
separate  rasters  are  projected,  the  signal  bandwidths  implied  are  easily 
handled  by  conventional  electronics,  modulators  and  line  scanners. 

Thus,  within  the  CIG  and  display  components,  development  problems  are 
reduced  mainly  at  the  expense  of  bulk  on  the  user's  helmet.  The  current 
estimate  of  weight  for  the  helmet,  based  on  weight  of  existing  galvanometers, 
some  detailed  optical  design  and  outline  design  of  mounting  structure,  is  2.5 
lbs . 


Some  CIG  development  will  be  required.  A  contract  has  been  awarded  to 
General  Electric,  Daytona  to  recommend  modifications  to  the  VTRS-CIG  to  allow 
it  to  interface  to  the  display  system.  These  modifications  include: 
capability  for  two  channels  to  select  and  process  different  levels  of  detail 
simultaneously  from  the  same  active  environment;  capability  to  Interface 
simultaneously  to  the  host  computer,  the  head  tracking  system  and  the  eye 
tracking  system;  capability  to  provide  distortion  correction  in  both  channels; 
capability  to  synchronize  output  video  with  a  roughly  synchronized  line 
scanner;  capability  to  blank  and  blend  inset  AOI  with  surrounding  IFOV;  and 
capability  to  provide  offset  signals  to  the  line  scan  offset  and  frame  scan 
galvanometers  to  follow  eye  movements  and  compensate  for  errors  due  to  rapid 
head  attitude  rates.  As  of  this  writing,  the  CIG  modifications  are  considered 
low  risk  developments. 

The  most  severe  residual  development  problems  are  considered  likely  to  be 
found  in  the  areas  of  head  tracking  and  eye  tracking.  Fast,  precise  head 
tracking  is  vital  to  any  helmet-mounted  system  for  stability  of  the  projected 
image.  While  eye-tracking  need  not  be  precise,  a  method  must  be  identified 
which  has  high  reliability,  which  can  be  set  up  rapidly,  and  which  is 
acceptable  to  pilots  -  probably  ruling  out  most  of  the  current  clinical 
techniques. 

The  present  status  of  the  NTEC  investigations  of  head-  and  eye-tracking 
are  described  below. 

8.  HEAD  TRACKING 


The  head  tracking  system  is  required  to  measure  yaw,  pitch  and  roll  of 
the  observer's  head  relative  to  the  simulated  cockpit  structure.  In  order  to 
determine  performance  requirements  on  the  head  tracking  system  a  helmet 
mounted  sight  (Polhemus  Spasyn  Helmet  Mounted  Sight  Model  IIIA)  has  been 
procured.  This  system  was  interfaced  to  the  VTRS  CIG  system  which,  in  turn, 
supplied  video  to  a  miniature  projection  CRT  mounted  on  a  helmet. 
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This  equipment  was  then  utilized  to  perform  a  subjective  evaluation 
experiment.  The  experiment  consisted  of  having  the  CIG  fly  along  a  canned 
flight  path  over  an  environment  consisting  of  an  airfield  and  surrounding 
terrain.  The  observer's  head  direction  determined  the  viewing  direction  used 
by  the  CIG  to  compute  the  scene. 

The  observed  defects  in  image  stability  were: 

a.  A  noticeable  angular  displacement  of  objects  in  the  scene  while  the 
observer's  head  is  rotating. 

b.  An  occasional  jitter  of  the  displayed  imagery  even  when  the  head 
tracking  system  sensor  was  fixed  relative  to  the  transmitter  and  cockpit. 

The  angular  displacement  with  observer's  bead  rotation  was  found  to  be 
directly  related  to  the  known  throughput  delay  of  the  head-tracking  and  CIG 
system.  The  total  delay  is  0.1  sec,  so  that  a  30°/sec  head  rotation  rate 
produces  a  3°  image  displacement.  This  effect  was  judged  to  be  objectionable 
and  an  effort  is  underway  to  correct  the  defect  by  dynamically  displacing  the 
raster.  The  amount  of  angular  displacement  is  made  equal  to  the  difference 
between  current  (or  most  recent)  measured  head  attitude  and  the  head  attitude 
utilized  to  compute  the  current  scene.  This  approach  causes  the  instantaneous 
field  of  view  to  be  reduced  by  the  amount  of  the  motion  compensating 
displacement  during  head  motion.  As  of  this  writing,  the  hardware  has  been 
fabricated  using  a  microprocessor  and  the  software  program  completed. 

However,  the  evaluation  was  not  yet  underway.  This  technique  should  eliminate 
the  effect  of  CIG  throughput  delay  at  the  expense  of  reduced  instantaneous 
field  of  view  during  head  motions.  The  effect  of  head  tracking  system 
throughput  delay  cannot  be  eliminated  but  may  be  reduced  utilizing  linear 
extrapolation.  The  capability  to  perform  linear  extrapolation  is  included  in 
the  microprocessor  and  will  be  evaluated  concurrently  with  the  CIG  throughput 
delay  compensation  scheme  outlined  above. 

The  magnitude  of  the  display  jitter,  using  the  current  head-tracking 
device,  was  approximately  15  milliradians .  Measurement  of  signals  indicated 
that  a  large  part  of  this  system  noise  originated  in  the  internal  components 
of  the  head  tracking  system.  As  of  this  writing,  the  manufacturer  has  agreed 
to  evaluate  the  system. 

An  in-house  analysis  of  alternative  electro-optic  techniques  for  head 
attitude  sensing  is  being  pursued.  For  use  in  a  simulator,  it  should  be 
feasible  to  improve  on  existing  head-trackers  which  were  designed  for  use  in  a 
real  cockpit  environment,  since  the  simulator  environment  permits  much  greater 
flexibility  in  design. 

9.  EYE  ATTITUDE  MEASUREMENT  SYSTEM 

The  eye  attitude  measurement  system  provides  the  CIG  with  the  viewing 
direction  needed  to  place  the  high  detail  area  of  interest  in  that  part  of  the 
projected  field  corresponding  to  the  observer's  central  vision.  The  eye 
direction  information  is  not  used  to  stabilize  the  resultant  display  but 
merely  to  select  the  area  within  which  high  detail  and  resolution  appear,  so 
that  the  measuring  device  may  have  relatively  poor  precision,  accuracy  and 
response.  Some  of  the  design  goals  for  the  eye  tracking  system  are:  (a)  to 
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be  visually  unobtrusive,  (b)  a  precision  of  +  2°,  (c)  an  accuracy  (relative  to 
head  pointing  direction)  of  +  2°  and,  (d)  a  response  time  of  less  than  16 
milliseconds.  Experiments  utilizing  a  one-axis  (azimuth  only)  limbus  tracker 
indicate  that  these  requirements  will  suffice  provided  that  the  width  of  the 
A01  is  25s  or  more,  and  the  response  time  (sum  of  eye  tracker  and  CIG  response 
times)  is  less  than  100  milliseconds.  (The  experiments  were  carried  t  using 
photographic  imagery,  with  variable  resolution  but  not  variable  detail,  so 
that  applicability  to  CIG  imagery  is  somewhat  questionable.) 

If  the  total  throughput  delay  is  too  large,  a  technique  utilizing 
saccadic  prediction  may  be  required.  Such  a  system  is  currently  being 
developed  under  a  contract  with  Carnegie-Mellon  University  (Dr.  Terry  Bahill  - 
Principal  Investigator).  Dr.  Bahill  is  also  Investigating  the  use  of 
electro-oculography  (EOG). 

Reference  3  provides  an  excellent  survey  of  eye  attitude  measurement 
techniques  and  limitations.  Ideally,  the  selected  method  for  eye  tracking 
will  require  no  attachments  to  the  subject's  head.  Although  remote 
oculometers  exist,  they  are  not  capable  of  measuring  eye  attitude  for  a  large 
range  of  head  rotations,  and  the  use  of  multiple  remote  oculometers  to  cover 
all  likely  head  attitudes  does  not  seem  practical.  The  least  obtrusive 
head-mounted  technique  for  eye  tracking  is  EOG.  This  method  requires 
electrodes  to  be  taped  to  the  subject's  face,  but  the  electrodes  are  not  found 
seriously  objectionable  and  they  do  not  have  significant  weight  or  obstruct 
the  subject's  vision  in  any  way. 

For  these  reasons,  EOG  is  considered  a  promising  method  for  use  in 
eye-coupled  flight  simulation  displays.  EOG  techniques  are  notorious  for 
drifting  but  drift  can  be  corrected  in  theory  by  automatic  recalibration 
utilizing  a  single  remote  oculometer  which  will  provide  a  reading  whenever  the 
observer  is  looking  within  10°  of  it.  Thus  a  combination  of  two  unobtrusive 
techniques  may  prove  feasible. 

The  eye-attitude  measuring  system  remains  a  high  risk  area.  However,  a 
system  which  has  no  eye  tracking,  in  which  the  AOI  remains  in  the  center  of 
the  head-tracked  IFOV,  is  considered  a  viable  alternative.  In  this  case,  the 
area  of  the  AOI  would  be  increased,  with  some  loss  of  resolution,  to  encompass 
most  normal  eye  rotations  with  respect  to  the  head. 

10.  SUMMARY  AND  CONCLUSIONS 


A  visual  simulation  system  has  been  described  which  takes  advantage  of 
human  visual  system  limitations  to  provide  a  display  which  will  be  perceived 
as  having  both  high  resolution  and  very  wide  angle,  utilizing  only  two 
display/image  generator  channels.  The  system  design  appears  feasible 
utilizing  available  technology  with  the  exception  of  two  high  risk  areas 
namely:  head  attitude  and  eye  attitude  measurement  systems. 
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ABSTRACT 

Design  of  visual  simulators  calls  for  consideration  of 
behavioral  components  so  that  one  may  better  match  display  and 
visual  system  characteristics  in  order  to  optimize  bandwidth 
and  data  base  allocation.  In  this  paper  we  consider  some  of  the 
well  documented  and  better  understood  aspects  of  visual  psycho¬ 
physical  capabilities,  with  special  emphasis  on  spatio-temporal 
frequency  analysis.  It  is  concluded  that  one  should  attempt, 
and  it  may  at  this  stage  be  feasible,  to  design  display  systems 
employing  an  exponential  spiral  raster  which  matches  the  infor¬ 
mation  density  in  the  display  to  the  information  handling  capa¬ 
bilities  of  the  visual  system. 
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SOME  PSYCHOPHYSICAL  ASPECTS  OF  VISUAL  PROCESSING  OF 
DISPLAYED  INFORMATION 


As  one  strives  to  generate  more  realistic  imagery  in  advanced 
simulators  for  pilot  training  (ASPT) ,  scene  complexity  and  require¬ 
ments  for  bandwidth  and  data  base  increase  tremendously  (Schachter 
and  Ahuja,  1980) .  Since  practical  limitations  impose  constraints 
on  the  attainable  scene  complexity,  it  becomes  crucial  to  attempt 
an  optimal  design  of  computer  generated  imagery  (CGI) .  Although 
it  is  not  immediately  clear  exactly  what  the  criteria  for  design 
optimality  ought  to  be,  certaiiily  such  criteria  will  depend  in  part 
on  the  information  processing  properties  of  the  human  visual  system. 

It  is  therefore  relevant,  regardless  of  the  approach  employed  in 
visual  simulation  and  the  technology  implemented,  to  consider  phy¬ 
siological  and  psychophysical  characteristics  so  as  to  match  the 
CGI  resources  to  the  inherent  signal  detecting  abilities  of  human 
observers.  In  this  paper  we  present  an  overview  of  some  aspects 
of  these  capabilities  of  the  visual  system,  with  special  emphasis 
on  the  dynamics  of  visual  signals,  spatio-temporal  modulation 
transfer  functions  for  target  detection,  and  some  considerations 
of  signal- to-noise  ratio. 

It  is  important  to  observe  that  the  visual  world  is  mapped  by 
the  retina  onto  dynamic  patterns,  even  during  visual  fixation  because 
of  the  constant  involuntary  movement  of  the  eye.  During  fixation 
the  visual-occulomotcr  system  still  generates  slow  drifts  lasting 
a  few  hundred  milliseconds,  separated  by  rapid  jumps  known  as  micro- 
saccades  having  high  velocities  and  amplitudes  of  a  few  minutes  of 
arc  (Boyce,  1967).  It  is  believed  that  there  is  an  elevation  of 
visual  threshold  and  a  suppression  of  pattern  information  processing 
during  these  microsaccades ,  as  during  larger  saccades  such  as  those 
which  subserve  foveation  in  visual  space  and  position  correction 
during  tracking  movements.  Thus  in  the  natural  mode  of  visual  infor¬ 
mation  processing,  the  world  is  necessarily  sampled  into  temporal 
frames  of  a  few  hundred  milliseconds.  Various  studies  indicate  that 
some  kind  of  weighted  response  integration  takes  place  over  such 
frames  (Koenderink  &  van  Doom,  1978;  Legge,  1978).  If  in  fact 
drift  duration  coincides  with  the  duration  of  the  processing  frame, 
then  there  should  be  a  dependency  of  the  drift  on  the  stimulus  temporal 
structure,  determined  by  the  time- frequency  uncertainty  principle 
AfAc  =  const  (Gabor,  1946).  One  of  us  (YYZ)  is  presently  investi¬ 
gating  possible  adaptive  characteristics  cf  intersaccadic  frame 
duration.  Considering  the  signal  detection  problem,  integration  of 
visual  signals  over  such  finite  frames  is  essential  in  order  to 
achieve  an  acceptable  signal- to-noise  ratio,  since  the  visual  system 
is  exposed  to  a  variety  of  sources  of  noise  that  contaminate  the 
image  quality.  Even  the  quantal  nature  of  light  imposed  some  con¬ 
straints  in  the  course  of  evolution  on  visual  system  design  (Rose,  1974) 
The  evolution  of  photoreceptors  achieved  the  theoretical  limit  of 
signal  detection,  since  retinal  rods  can  respond  to  single  photons. 
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As  a  consequence  of  this  optimal  design,  however,  photon  noise 
induces  signal  variability.  Secondly,  since  the  energy  of  a 
single  photon  is  far  less  than  that  of  any  appreciable  bioelectric 
signal,  an  intermediate  process  of  amplification  is  necessarily 
incorporated  into  photoreceptor  design.  This  process  introduces  an 
internal  source  of  noise.  Consequently,  about  5-15  photons  are 
required  for  a  photic  stimulus  to  be  detected  reliably.  In  addition, 
there  are  in  the  visual  system  various  other  sources  of  neural  noise. 
The  interplay  of  all  of  these  sources  of  noise  determines,  for 
example,  the  psychophysical  increment  intensity  thresholds.  Theor¬ 
etical  analysis  indicates  that  due  to  photoreceptor  nonlinearity, 
quantal  noise  is  the  main  limiting  factor  of  incremental  intensity 
sensitivity  only  over  an  intermediate  (mesopic)  range  of  light 
intensities.  At  low  and  high  intensities,  receptor  and  neural  noise 
determine  the  bound  on  sensitivity  (Zeevi  &  Mangoubi ,  1978).  The 
noise  characteristics  of  the  human  visual  system  are  relevant  for 
CGI  both  in  terms  of  an  observer's  ability  to  resolve  "barely  visible" 
contrast  in  space/time,  and  in  terms  of  how  noise-free  a  display 
system  must  be  in  order  to  be  perceptually  realistic. 

Although  image  bluring  would  be  an  expected  consequence  of  eye 
movements  during  integration  time,  visual  acuity  experiments  actually 
indicate  that  artificial  image  stabilization  does  not  improve  acuity 
whereas  increasing  the  signal  duration  up  to  about  300  msec  does 
(Keesy,  1960).  Thus,  such  experiments  indicate  that  at  some  stage 
in  the  processing  hierarchy  there  should  exist  a  mechanism  for 
separation  of  spatial  and  temporal  information.  One  model  assumes 
that  this  operation  takes  place  in  the  frequency  domain,  arguing 
that  in  the  early  stages  of  preprocessing,  visual  signals  are 
decomposed  into  spatio-temporal  harmonic  frequency  components  and 
that  subsequently  these  are  grouped  and  summed  in  the  frequency 
domain  to  provide  a  "frozen  picture"  (Fig.  1)  or  for  the  extraction 
of  temporal  or  velocity  information  (Gafni  & Zeevi ,  1977  ,  1979  ; 

Zeevi  &  Gafni,  1980).  In  short,  if  we  consider  an  idealized  expansion 


Fig.  1.  Grouping  and  summation  of  spatio-temporal  harmonic  com¬ 
ponents  to  provide  a  "frozen  picture".  Segregation  of  components 
is  by  spatial  frequency;  summation  operates  across  temporal  frequencies. 
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into  three-dimensional  harmonic  components,  the  extraction  of 
spatial  information  from  a  dynamic  pattern  can  be  formally  presented 
by 

f  (x,y ,  0)  =  F  2{/F3  [f  (x,y,t)  ]du)t> 

3  -2 

where  F  represents  the  three-dimensional  Fourier  transform  and  F 

the  inverse  two-dimensional  transform.  This  model  brings  us  to  an 

important  and  somewhat  controversial  facet  of  vision  research. 

In  engineering  we  often  prefer  to  characterize  systems  in  the 
frequency  domain  employing  transfer  functions.  This  approach  was 
found  to  be  useful  also  in  optics  and  subsequently  in  vision  research 
where  spatial  and  temporal  frequencies  have  been  widely  used  by 
various  investigators  (e.g.  Campbell  &  Robson,  1964;  Kelly,  1961). 

The  global  envelope  of  human  visual  system  sensitivity,  represented 
in  Fig.  2,  specifies  the  total  range  of  spatio-temporal  modulation 
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Fig.  2.  Global  envelope  of  spatio-temporal 
sensitivity  of  human  visual  system. 

(From  Kelly,  1977.) 


Fig.  3.  Human  sensitiv¬ 
ity  to  uniform  flicker. 
Peak  sensitivity  shifts 
to  higher  frequencies 
at  higher  levels  of  mean 
luminance.  (From  Kelly, 
1961. ) 
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that  can  convey  visual  information.  Any  image  information  involv¬ 
ing  modulation  outside  of  this  spatio-temporal  envelope,  or  on  low 
points  on  the  surface,  in  general  may  not  be  detectable.  A  number 
of  gross  constraints  for  CGI  present  themselves  immediately  from 
Fig.  2:  First,  CGI  displays  based  on  television  techniques  employ¬ 
ing  raster  scans  must  have  frame  rates  exceeding  roughly  50  Hz  in 
order  for  the  frame  flicker  to  be  imperceptible.  The  critical 
flicker  fusion  frequency  depends,  however,  on  many  variables. 

(i)  As  may  be  clearly  seen  in  Fig.  2,  for  the  lower  spatial  frequencies 
the  temporal  sensitivity  extends  to  higher  flicker  frequencies  than 
is  the  case  for  higher  spatial  frequencies.  (ii)  The  spatio- 
temporal  detection  envelope  changes  in  shape  considerably  as  a 
function  of  eccentricity,  the  distance  on  the  retina  away  from 
the  fovea.  The  surface  shown  in  Fig.  2  applies  only  for  foveal 
vision  (roughly  the  central  two  degrees).  Eccentric  vision  is 
worse  in  detecting  all  spatial  frequencies  than  foveal  vision, 
but  temporal  modulation  of  peripheral  information  significantly 
enhances  its  detectability.  (iii)  The  mean  luminance  level  at 
which  image  contrast  is  viewed  also  has  a  considerable  effect  on 
the  shape  of  the  spatio-temporal  envelope  and  the  locations  of 
its  maxima;  Fig.  3  shows  flicker  sensitivity  curves  at  a  range 
of  luminance  levels  spanning  five  log  units.  The  higher  the  mean 
luminance,  the  higher  the  flicker  frequency  that  can  still  be 
resolved  without  fusion.  (iv)  The  mean  luminance  level  also 
affects  the  peak  of  the  purely  spatial  modulation  transfer  func¬ 
tion,  as  in  the  temporal  case:  the  higher  the  mean  luminance, 
the  higher  the  spatial  frequency  of  maximum  sensitivity.  At  low 
(scotopic)  luminance  levels,  non-foveal  regions  of  visual  space 
are  in  fact  the  most  sensitive  areas  for  low  and  intermediate 
spatial  frequencies.  These  four  characteristics  are  readily 
recognizable  in  the  common  experience  of  viewing  ordinary  tele¬ 
vision  in  one's  visual  periphery,  with  a  perceptible  flicker  that 
is  not  noticeable  in  foveal  viewing;  and  in  the  well-known  loss 
in  visual  acuity  with  eccentric  viewing  or  with  reduced  ambiant 
luminance . 

In  order  to  make  optimal  use  of  bandwidth  resources  in  CGI, 
its  allocation  to  spatio-temporal  modulation  should  not  exceed 
the  detection  limits  of  the  human  visual  system  as  functions  of 
eccentricity,  mean  luminance,  and  the  limiting  tradeoffs  between 
spatial  sensitivity  and  temporal  sensitivity.  The  economies 
resulting  from  these  considerations  can  be  very  large,  since  the 
psychophysical  magnitudes  of  the  dependencies  mentioned  above 
typically  exceed  a  ten-fold  range.  Particularly  significant  is 
the  dependence  of  resolution  on  eccentricity,  since  pixel  density 
requirements  translate  directly  into  CGI  bandwidth  requirements. 

As  seen  in  Fig.  2,  the  disappearance  of  spatial  frequency  contrast 
sensitivity  above  roughly  20  cycles  per  degree  (cpd)  for  foveal 
vision  implies  that  pixel  densities  in  central  viewing  need  not 
exceed  1.5  minutes  of  arc,  which  corresponds  to  a  pixel  size  of 
0.5  millimeter  when  viewed  from  a  distance  of  one  meter  in  order 
for  the  discreteness  of  the  image  to  be  unresolvable  even  when 
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adjacent  pixels  are  at  maximum  contrast.  (In  this  estimage  we 
do  not  consider  the  phenomenon  of  hyperacuity,  in  which  vernier 
separations  as  small  as  4  seconds  of  arc  may  be  resolved,  because 
this  capability  arises  from  the  detection  of  phase  information 
independent  of  spatial  frequency  (Westheimer,  1979).)  The  fall-off 
in  visual  acuity  with  increasing  eccentricity  from  the  fovea  is 
dramatic:  for  spatial  frequencies  in  the  range  of  0.25  to  2  cycles 

per  degree,  the  contrast  thresholds  increase  by  a  factor  of  two  for 
every  12  degrees  in  eccentricity  (Koenderink,  Bouman,  de  Mesquita,  & 
Slappendel,  1978),  and  for  any  given  spatial  frequency  the  visual 
field  is  blind  beyond  a  certain  critical  eccentricity.  The  critical 
eccentricity  decreases  monotonically  as  a  function  of  spatial  fre¬ 
quency;  thus,  for  example,  a  spatial  frequency  of  2  cpd  becomes 
invisible  at  an  eccentricity  of  30°;  5  cpd  becomes  invisible  at  20°; 

10  cpd  becomes  invisible  at  15°;  and  16  cpd  becomes  invisible  at  6° 
(Koenderink  et  al. ,  1978).  Thus  if  one  were  to  generate  a  pixel- 
based  display  system  in  such  a  way  as  to  have  pixel  size  be  specified 
by  the  maximum  detectable  spatial  frequency  for  each  eccentricity, 
then  at  a  viewing  distance  of  one  meter  the  pixel  size  should  vary 
monotonically  with  eccentricity  from  0.5  mm  in  the  fovea  to  2  mm  at 
an  eccentricity  of  20°  and  0.5  centimeter  at  an  eccentricity  of  30°. 
This  ten-fold  reduction  in  required  pixel  density  over  the  first  30° 
in  eccentricity  implies  likewise  a  ten-fold  reduction  in  CGI  band¬ 
width  allocation  as  a  function  of  eccentricity.  The  potential  band¬ 
width  reduction  is  very  substantial:  a  one-degree  wide  "ring" 
centered  on  the  fovea  with  a  radius  of  50  degrees  in  a  conventional 
constant-pixel-density  display  consumes  300  times  as  much  bandwidth 
as  does  a  one-degree  patch  in  the  critical  fovea,  but  94  %  of  the 
bandwidth  is  wasted  since  such  an  image  is  indistinguishable  for 
human  observers  from  one  whose  pixel  width  at  this  eccentricity 
is  16  times  greater. 

Because  the  function  relating  the  required  pixel  size  to  the 
eccentricity  is  a  radial  function,  it  may  be  most  efficient  to 
design  CGI  displays  with  spiral  rather  than  Cartesian  raster  systems. 
Current  rapid  advances  in  unobtrusive  eye-position-tracking  technol¬ 
ogies  (e.g.  IR  beam,  magnetic  coil,  CCD  camera)  make  it  possible  in 
principle  to  track  the  location  of  foveal  viewing  and  to  use  this 
real-time  information  to  define  the  origin-of-coordinates  in  CGI 
displays  for  parameters  such  as  pixel  density  that  depend  on  eccen¬ 
tricity.  Generating  images  in  polar  rather  than  rectangular  coor¬ 
dinates  is  technically  easy  to  do,  since  the  required  X-  and  Y-axis 
deflection  signals  for  CRT  plates  require  only  a  quadrature  phase- 
split  pair  of  sinusoids  and  an  exponential  ramp  function  to  produce 
the  log  spiral  wave  forms :  . 

V  (t)  =  (e-1) cos (wt) 

x 

vy(t)  =  (et-l) sin(wt) 

(The  exponential  ramp  would  also  have  to  multiply  the  beam  current 
in  order  to  maintain  a  constant  phosphor  writing  speed  independent 
of  radius  to  keep  luminance  constant;  and  it  might  be  necessary  to 
add  raster  noise  proportional  to  radius  to  smear  out  the  separation  of 
raster  lines  at  large  eccentricities.)  A  constant  rate  of  informa- 
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tion  flow  into  a  spiral  raster  display  produces  effectively  a 
"polar  pixel"  whose  solid  angular  subtence  is  proportional  to 
eccentricity  (since  the  rotary  velocity  of  the  beam  is  propor¬ 
tional  to  eccentricity  and  raster  density  is  inversely  so) ,  and 
this  property  automatically  matches  the  fall-off  in  resolution 
of  the  visual  system  reciprocally  with  eccentricity.  (One  of  us 
(JGD)  has  designed  and  used  such  spiral  raster  displays  for  psycho¬ 
physical  purposes.)  Thus  a  log  spiral  raster  automatically  converts 
a  constant  rate  of  information  flow  from  CGI  hardware  into  a  display 
whose  information  density  more-or-less  matches  the  decline  in  visual 
acuity  with  radius.  The  desired  distribution  of  information  density 
would  be  roughly  as  represented  in  Fig.  4,  as  adapted  from  Koenderink 
et  al.'s  (1978)  model  for  the  size  distribution  of  retino-cortical 
units.  Their  careful  measurements  of  spatial  frequency  sensitivities 
as  function  of  eccentricity  suggest  that  when  a  single  scaling  factor 
compensates  for  eccentricity  by  multiplying  both  the  spatial  frequency 
content  of  a  target  and  its  extent,  then  the  spatio-temporal  contrast 
thresholds  are  identical  across  the  entire  visual  field. 


Fig.  4.  Proposed  spatial  distribution 
of  information  density  in  CGI  displays 
to  match  human  visual  resolution  for 
optomizing  CGI  bandwidth  allocation. 
(From  Koenderink  et  al.,  1978.) 


It  is  generally  believed  today  that  the  global  modulation 
transfer  functions  discussed  so  far  (at  various  eccentricities 
and  luminance  levels)  are  in  fact  just  "envelopes"  of  more  sharply 
defined  underlying  mechanisms.  This  so-called  "channels"  hypothe¬ 
sis  has  been  especially  articulated  with  regard  to  spatial  frequency 
mechanisms.  In  part  the  model  is  supported  by  physiological  measure 
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ments  of  the  response  properties  of  single  neurons  in  the  visual 
cortex  of  cat  and  monkey.  Many  investigators  have  reported  that 
neurons  in  cortical  Area  17  respond  to  the  visual  world  in  a  way 
that  is  localized  both  in  space  and  in  spatial  frequency,  in  the 
sense  that  the  "best"  stimulus  for  evoking  large  responses  from 
single  neurons  in  terms  of  nerve  impulse  frequency  is  a  stimulus 
that  exists  in  a  certain  well-defined  region  of  visual  space  (the 
"receptive  field"  of  the  cell  in  question)  and  is  also  localized 
in  spatial  frequency  by  a  certain  periodicity  of  luminance  modulation. 
Thus  a  stimulus  becomes  "irrelevant"  to  a  given  cell  either  by  its 
being  in  the  wrong  location  in  visual  space  or  by  its  having  the 
wrong  spatial  frequency  of  contrast  modulation.  These  joint  char¬ 
acteristics,  of  retinotopic  mapping  of  visual  space  and  spatial 
frequency  tuning,  necessarily  require  a  trade-off  in  localizability 
in  the  two  domains  (just  as  the  time-frequency  tradeoff  mentioned 
earlier) .  The  response  characteristics  of  cortical  neurons  are 
conveniently  described  in  terms  of  a  receptive  field  profile,  which 
is  a  function  over  visual  space  specifying  the  excitatory  or  inhibi¬ 
tory  influence  that  light  has  on  a  cell.  Typically  such  measured 
profiles  have  the  appearance  shown  in  Fig.  5  :  one  or  two  regions 
in' which. light  excites  the  firing  rate  of  the  cell,  interlea\ed  by 
one  or  two  regions  in  which  light  inhibits  the  cell,  and  beyond 
these  regions  light  has  no  influence  on  the  cell.  It  is  obvious 
that  trying  to  increase  the  spatial  resolution  of  such  mechanisms  by 
making  them  smaller  would  destroy  their  spatial  frequency  tuning, 
since  the  limiting  case  of  an  infinitesimal  detector  would  respond 
equally  well  to  all  spatial  frequencies.  Likewise  it  is  obvious  that 
increasing  their  spatial  frequency  tuning  by  adding  more  periods 
of  excitatory/ inhibitory  regions  would  occupy  a  large  region  of 
visual  space  and  thus  destroy  spatial  resolution.  The  particular 


Fig.  5.  Gabor  elementary  signals  possessing  localization 
in  both  the  space  domain  and  the  spatial  frequency  domain. 

The  product  of  the  "spread"  in  these  two  domains  is  minimal 
for  the  functions  shown,  which  are  sine  and  cosine  multiplied 
by  a  Gaussian.  The  functions  form  a  complete  basis  set. 
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functions  shown  are  the  product  of  a  sinusoid  or  cosinusoid  and 
a  Gaussian;  Gabor (1946)  proved  that  such  functions  minimize  the 
product  of  their  "spread"  in  the  space  domain  and  the  spatial 
frequency  domain.  With  both  the  even  and  odd  members  (sine  and 
cosine)  included,  the  functions  form  a  basis.  Mathematically,  they 
are  described  by 

f (x)  =  e“(x/X)2  -  ~Wx-ie 


-ico' x  i6 
e  e 


for  spatial  frequency  a. spatial  extent  A,  and  spatial  phase  6 
which  is  0  for  even  symmetry  and  tt/2  for  odd  symmetry.  The  descrip¬ 
tion  of  these  waveforms  in  the  spatial  frequency  domain  may  be 
obtained  by  the  Fourier  transform  of  f(x) ,  which  is 

2  2 

u> * )  A~  i6 

F  ( oi)  =  e  e 

from  the  Modulation  Theorem.  It  is  seen  that  the  "spread"  in  the 
space  domain  is  A  while  the  "spread"  in  the  spatial  frequency 
domain  is  1/A,  so  sharpening  up  the  resolution  in  one  domain  bears 
a  proportionate  price  in  the  other  domain.  The  fact  that  the  visual 
system  neurons  evidently  balance  this  trade-off  by  possessing  relative 
spatial  retinotopic  localization  and  at  the  same  time  a  spatial 
frequency  bandwidth  of  about  one  octave  with  matched  sine  and  cosine 
(phase  quadrature)  cell  pairs  (Pollen  &  Ronner,  in  press)  is 
suggestive  that  important  kinds  of  visual  processing  are  goiny  on 
in  both  domains. 
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Fig.  6.  Global 
spatial  modula¬ 
tion  transfer  function 
in  the  Fourier  spatial 
plane. 
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Psychophysical  evidence  for  visual  processing  in  the  spatial 
(and  temporal)  frequency  domains  arises  from  the  observation  that 
a  high  contrast  image  containing  only  a  "singletone"  of  spatio- 
temporal  contrast  modulation  is  capable  of  significantly  lowering 
the  sensitivity  to  similar  "test"  images  presented  either  simul¬ 
taneously  (masking)  or  subsequently  (adaptation) .  When  the  masking 
or  adapting  stimulus  has  a  spectral  content  which  differs  from  that 
of  the  test  stimulus,  it  is  possible  to  estimate  a  "bandwidth"  of 
the  threshold  elevation  effect  in  a  parametric  way.  An  excellent 
review  of  this  broad  literature  is  found  in  Sekuler  (1974)  .  The 
results  of  such  experiments  generally  yield  estimates  of  approxi¬ 
mately  one  octave  for  the  spatial  frequency  bandwidths  of  the 
underlying  "channels,"  as  well  as  an  orientation  bandwidth  of 
approximately  15  degrees.  If  we  consider  for  the  moment  just  the 
spatial  aspects  of  vision  apart  from  the  temporal,  it  becomes  clear 
that  the  two-dimensional  manifold  of  visual  space  has  a  2d  (not 
just  a  ID)  spatial  frequency  characteristic .  It  is  useful  to 
parse  this  spatial  Fourier  plane  in  polar  coordinates,  so  that  the 
radial  coordinate  is  spatial  frequency  and  the  angular  coordinate 
is  orientation.  Then  the  global  modulation  transfer  function  of 
the  human  visual  system  is  found  empirically  to  be  roughly  as  shown 
in  Fig.  6.  The  overall  bandpass  character  of  spatial  contrast 
modulation  sensitivity  is  reflected  in  the  depressed  center,  indi¬ 
cating  both  low  and  high  spatial  frequency  attenuations  in  sensitivity. 
The  volcano's  anisotropy  reflects  the  well-known  "oblique  effect", 
by  virtue  of  which  the  vertical  and  horizontal  orientations  are 
more  readily  detected  than  the  oblique  orientations.  This  effect 
is  far  more  pronounced  at  higher  spatial  frequencies  than  at  low; 
at  around  10  cycles  per  degree,  the  anisotropy  may  amount  to  a 

three-fold  difference  in  sensitivity  at  vertical  versus  oblique 
orientations.  The  global  surface  shown  in  Fig.  6  can  be  broken 
down  into  component  2D  spatial  frequency  channels  by  studying 
masking  interference  effects.  The  resulting  threshold  elevation 
surface  shown  in  Fig.  7  reveals  the  kind  of  spatial  frequency  / 
orientation  mechanism  that  a  "channel"  is  thought  to  be,  namely 
a  spectral  filter  with  roughly  a  one  octave  spatial  frequency 
bandwidth  and  about  15  degrees  orientation  bandwidth.  The  spectral 
shape  shown  can  serve  as  a  sort  of  guideline  for  the  conditions 
under  which  two  different  spatial  targets  can  mask  each  other  or 
in  other  ways  interfere  as  a  result  of  being  processed  by  the  same 
frequency/orientation  channel.  As  a  rule,  the  spatial  frequency 
bandwidth  is  proportional  to  the  channel's  center  frequency, 
falling  generally  within  an  octave  on  either  side.  A  channel's 
orientation  bandwidth  appears  to  be  independent  of  what  its  pre¬ 
ferred  spatial  frequency  is.  Within  any  particular  channel, 
however,  the  orientation  tuning  and  spatial  frequency  tuning 
are  not  separable  variables,  and  the  band  of  orientations  over  which 
a  channel  responds  may  shrink  in  size  by  a  factor  of  two  when 
driven  at  non-optimal  frequencies  (Daugman,  1980a) .  The  polar 
spectral  nonseparability  of  visual  channels  is  predictable  from 
considerations  of  neural  receptive  field  profiles  (Daugman,  1980b 
&  1981a) . 
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Fig.  7.  The  spectral  shape  of  a  tv/o-dircnsionni  spatial 
frequency  channel  in  the  human  visual  system,  as  inferred 
from  spread-of-masking  threshold  elevation  experiments. 


The  finite  bandwidths  of  underlying  spatial  frequency  channels 
makes  it  possible  for  information  presented  in  closely  adjacent 
frequency  bands  to  interfere  or  mask  mutually.  This  subject  has 
been  carefully  explored  by  Julesz  &  colleagues  (1980)  regarding 
the  masking  effects  of  digital  quantization  of  images.  When  an 
image  is  discretized  into  a  coarse  density  of  grey  areas,  it  becomes 
very  difficult  to  recognize  when  the  density  is  only  about  10x10. 

By  selectively  filtering  out  the  spectral  components  introduced 
by  the  discretization,  Julesz  (1980)  showed  that  the  recognition 
problem  results  from  spatial  frequency  masking  of  the  low  frequency 
components  by  the  intermediate  frequencies  arising  from  discretiza¬ 
tion.  The  high  frequencies  were  not  responsible  for  the  masking 
effect,  presumably  because  their  spectral  remoteness  ensures  that 
they  would  be  processed  by  separate  mechanisms  in  the  visual  system. 
As  a  general  rule,  it  should  be  born  in  mind  in  CGI  design  that 
spatial  information  presented  in  spectral  bands  that  are  separated 
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by  more  than  one  octave  in  frequency  or  more  than  about  15  degrees 
in  orientation  will  be  processed  independently  by  the  visual  system, 
while  information  bands  spreading  within  a  channel  may  be  vulnerable 
to  mutual  masking.  The  relevance  of  such  masking  may  be  gauged  by 
the  principle  emerging  from  Ginsburg's  extensive  studies  (1978)  of  form 
vision  with  filtered  images,  that  primarily  the  low  spatial  frequencies 
in  an  image  are  used  for  form  identification  and  object  recognition. 

Until  recently  it  was  less  known  what  the  temporal  structure 
of  channels  is.  The  assumption  was  that  there  exists  only  a  bimodal 
partition,  into  "sustained"  and  "transient"  mechanisms  (Tolhurst,  1975) . 
Recent  experiments  employing  a  broad  repertoire  of  psychophysical 
techniques  indicate  however  that  there  exist  at  least  several  tem¬ 
poral  frequency  tuned  mechanisms  (Gafni  &  Zeevi  1980a, b;  Tyler,  1980; 
Zeevi  &  Gafni  1980) .  Experiments  on  masking  with  noise  indicate 
a  bandwidth  of  about  one  octave  for  the  entire  range  of  central 
frequencies  1-40  Hz  studied  with  this  technique  (Tyler,  1980) , 
whereas  results  of  experiments  based  on  subthreshold  summation  indi¬ 
cate  a  somewhat  narrower  bandwidth  of  2  to  3  Hz  for  the  range  of 
central  frequencies  explored,  5  to  10  Hz  (Gafni  &  Zeevi,  1980a). 

In  these  experiments,  stimuli  consisted  of  signals  which  match  in 
spatial  frequency  (0.5  or  1.0  cpd)  but  differ  in  temporal  frequency. 

The  temporal  frequency  of  one  of  the  signals,  f ^ ,  was  either  5,  7.5, 
or  10  Hz  and  its  contrast  was  adjusted  to  75  %  of  its  threshold 
value.  The  frequency  of  the  second  signal  was  f2  =  f,  +  Af,  where 
Af  is  a  variable  quantity  selected  to  yield  a  constant  phase  rela¬ 
tionship  between  the  flicker  and  its  beating  envelope  (Fig.  8) . 


Fig.  8.  The  sum  of  two  temporal  frequencies 
with  flicker  and  beating  envelope  being  in  phase. 

The  results  of  these  subthreshold  summation  estimates  are  in 
Fig  9.  It  is  interesting  that  whatever  frequency  is  selected  as  f^. 


Fig.  9.  Sub¬ 
threshold  summation 
data  for  the  wave¬ 
forms  in  Fig.  8, 
demonstrating  the 
existence  of  narrow- 
band  temporal 
channels. 
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a  channel  is  found  to  be  centered  there.  One  possible  explanation 
could  be  that  the  density  of  channels  is  very  high,  as  in  audition, 
with  extensive  overlap,  but  this  was  ruled  out  by  frequency  dis¬ 
crimination  experiments.  Thus  it  appears  as  though  there  are  only 
a  few  channels  with  adaptive  characteristics.  If  in  fact  there 
exists  an  adaptive  mechanism  which  maximizes  response  sensitivity, 
a  channel  would  adjust  its  central  frequency  so  as  to  maximize  its 
output  for  a  given  input  signal  combination.  Correcting,  accordingly, 
the  raw  data,  one  obtains  even  narrower  bandwidths  as  shown  by  the 
dashed  lines  in  Fig.  9. 

Preliminary  experiments  were  conducted  at  the  Human  Resources 
Laboratory  of  Williams  Air  Force  Base,  to  determine  and  compare 
the  masking  effects  of  static  and  dynamic  noise  on  target  detection 
and  tracking  as  assessed  Ly  probability  of  correct  responses  and 
their  reaction  time  (Zeevi,  1980).  Data  such  as  those  shown  in 
Fig.  10  demonstrate  that  as  target  luminance  and  the  corresponding 
contrast  decrease,  reaction  time  measured  as  saccadic  latency 
increases.  This  is  consistent  with  previous  findings  (Wheeless  et 
al . ,  1967)  and  should  be  taken  into  consideration  in  the  design  o7 
HTsplay  systems  for  ASPT.  the  effect  of  target  luminance  on  sac¬ 
cadic  latency  is  further  substantiated  once  masking  noise  is  intro¬ 
duced.  This  too  is  consistent  with  previous  results  demonstrating 
that  noise  masking  is  more  effective  than  coherent  maskina  at  the 
same  contrast  level  (Stromeyer  and  Julesz,  1972;  Kronauer'and 
Stromeyer,  1980).  More  interesting  though  are  the  findings  indi¬ 
cating  that  dynamic  noise  has  a  significantly  more  detrimental  effect 
on  target  detection  performance  than  static  noise  of  the  same  spatial 
spectral  content.  These  findings  further  support  our  argument  that 
dynamic  signals  are  preprocessed  in  the  visual  system  as  a  whole 
before  being  separated  into  spatial  and  temporal  information. 


COO). 


i 

s 


S’ 

s 

a 


1 

s 


500 


400  l 


300 


•  -  nolsa-fraa 
K  -  atatic  nois* 
q  -  dynamic  nolam 


U.O 

Lunin an ca  (ft.-laobart) 

Fig.  10.  F.xanplcs  of  saccadic  latencies 
in  detection  and  tracking  of  aperiodic 
point  target  subtending  about  0.4°  visual 
angle.  273 


soo  < — 


«.< 


To  sum  up,  we  have  discussed  a  few  aspects  of  visual  psycho¬ 
physics  relevant  to  the  perception  of  displayed  information.  We 
examined  signal- to-noise  ratio,  considering  both  internal  and 
external  forms  of  noise  and  its  masking  effects.  The  consequences 
of  constant  eye  movements  were  discussed  in  terms  of  a  temporal 
"framing"  of  the  visual  world,  which  is  a  crucial  part  of  cortical 
information  processing.  The  inherent  trade-off  between  time  and 
frequency  resolution  in  such  a  scheme  was  a  recurring  theme 
throughout  the  paper.  A  convenient  approach  was  to  employ  spatio- 
temporal  frequency  domain  analysis,  which  is  complementary  to  more 
traditional  space-time  analysis,  but  which  captures  more  powerfully 
the  visual  characteristics  that  are  relevant  to  engineering  consider 
ations.  Both  domains  are  really  necessary  for  a  complete  repre¬ 
sentation  of  visual  system  characteristics.  We  discussed  the 
global  envelope  of  spatio-temporal  sensitivity  and  the  evidence 
that  this  envelope  is  comprised  really  of  many  relatively  more 
narrowly  tuned  mechanisms.  As  reviewed  from  the  literature,  these 
spatio-temporal  characteristics  depend  dramatically  on  eccentricity 
and  mean  luminance.  For  reasons  of  optimal  bandwidth  allocation 
these  considerations  should  be  taken  into  account  in  display 
disign.  Although  for  these  reasons  it  appears  to  be  attractive 
to  generate  displays  using  exponential  spiral  rasters,  rather  than 
Cartesian  raster  scans,  it  should  be  born  in  mind  that  there  remain 
problems  in  superimposing  the  display  origin  of  coordinates  on 
the  visual  axis.  Achieving  this  will  require  high  precision  measure 
ment,  computation,  and  prediction  of  eye  position  end  point  during 
the  time  course  of  saccades.  Such  instrumentation  and  algorithms 
are  now  beginning  to  appear  feasible. 


274 


References 


1)  Boyce,  P.R.  (1967) ,  Monocular  Fixation  in  Human  Eye  Movement, 
Proc.  Roy.  Soc .  B  167,  293. 

2)  Campbell,  F.W.  and  Robson,  J.G.  (1969),  Application  of  Fourier 
Analysis  to  the  Modulation  Response  of  the  Eye,  J.  Opt.  Soc. 

Am.  54,  581. 

3)  Daugman,  J.G.  (1980a),  Two-Dimensional  Fourier  Analysis  of 
Visual  Channels  and  Cortical  Receptive  Fields,  Optical  Society 
of  America,  Sarasota,  April  1980. 

4)  Daugman,  J.G.  (1980b) ,  Two-Dimensional  Spectral  Analysis  of 
Cortical  Receptive  Field  Profiles,  Vision  Research  20,  847. 

5)  Daugman,  J.G.  (1981a) ,  Formal  Properties  of  Anisotropic  Linear 
Cortical  Mechanisms,  in  press,  Vision  Research. 

6)  Daugman,  J.G.  (1981b),  Spatial  Channels  in  the  Fourier  Plane, 
in  preparation. 

7)  Gabcr,  D.  (1946) ,  Theory  of  Communication,  U.  Inst.  Elect. 

Eng's,  93,  429. 

8)  Gafni,  H.  and  Zeevi,  Y.Y.  (1977) ,  A  model  for  Separation  of 
Spatial  and  Temporal  Information,  Biol.  Cybernetics  28,  73. 

9)  Gafni,  H.  and  Zeevi,  Y.Y.  (1979) ,  A  Model  for  Processing  of 
Movement  in  the  Visual  System,  Biol.  Cybernetics  32,  165. 

10)  Gafni,  H.  and  Zeevi,  Y.Y.  (198Ca) ,  Temporal  Frequency  Channels 

in  the  Visual  System,  Technion,  EE  Publication  No.  394. 

11)  Gafni,  H.  and  Zeevi,  Y.Y.  (1980b),  Temporal  Channels  in  Vision, 

Annual  Meeting  Optical  Society  Am.,  Chicago. 

12)  Ginsberg,  A.P.  (1978) ,  Visual  Information  Processing  based  on 
Spatial  Filters  Constrained  by  Biological  Data,  Ph.D.  Thesis, 
Cambridge  University. 

13)  Julesz,  B.  (1980),  Spatial  Frequency  Channels  in  One-,  Two-, 
and  Three-Dimensional  Vision:  Variations  on  a  Theme  by  B^k^sy, 
in  Visual  Coding  and  Adaptability  (C.  Harris,  ed.),  Erlbaum. 

14)  Keesy,  V.T.  (1960) ,  Effects  of  Involuntary  Eye  Movements  on 
Visual  Acuity,  J.  Opt.  Sec.  Am.  50,  769. 

15)  Kelly,  D.H.  (1961) ,  Visual  Responses  to  Time  Dependent  Stimuli: 
I.  Amplitude  Sensitivity  Measurements,  J.  Opt.  Soc.  Am.  51,  422. 

16)  Kelly,  D.H.  (1977) ,  Visual  Contrast  Sensitivity,  Optica  Acta, 

24,  107. 


275 


17)  Koendrerink,  J.  J.  and  Van  Doom,  A.  J.  (1978)  ,  Detectability 
of  Power  Fluctuations  of  Temporal  Visual  Noise,  Vision  Res 
18,  191. 

18)  Koenderink,  J.J.,  Bouman,  M.A.,  Bueno  de  Mesquita,  A.E.  and 
Slappendel,  S.  (1978),  Perimetry  of  Contrast  Detection  Thres¬ 
holds  of  Moving  Spatial  Sine  Wave  Patterns.  II.  The  Far  Peri¬ 
pheral  Visual  Field  (eccentricity  0°  -  50°)  J.  Opt.  Soc. 

68,  850. 

19)  Koenderink,  J.J.,  Bouman,  M.A.,  Bueno  de  Mesquita,  A.E.  and 
Slappendel,  S.  (1978),  Perimetry  of  Contrast  Detection  Thres¬ 
holds  of  Moving  Spatial  Sine  Wave  Patterns.  I.  The  Near  Peri¬ 
phery  Visual  Field  (eccentricity  0°  -  8°),  J.  Opt.  Soc.  Am. 

68,  845. 

20)  Kronauer,  R.E.  and  Stromeyer,  C.F.  (1980),  Comparison  of 
Masking  with  Noise  and  Coherent  Signals  (Personal  Communication) . 

21)  Legge,  G.  (1978) ,  Sustained  and  Transient  Mechanisms  in  Human 

Vision:  Temporal  and  Spatial  Properties,  Vision  Res.,  18,  69. 

22)  Pollen,  D.A.  Ronner,  C.F.  (1981),  Personal  Communication, 

Paper  submitted  to  Science. 

23)  Rose,  A.  (1974),  Vision:  Human  and  Electronics,  Plenum  Press, 
N.Y. 

24)  Schacter,  B  and  Ahu.ia,  N.  (1980)  "A  History  of  Visual  Flight 
Simulators,  Computer  Graphics  World,  3,  16. 

25)  Sekular,  R.  (1974),  Spatial  Vision,  Am.  Rev.  Psycho.,  25,  195. 

26)  Stromeyer,  C.F.  and  Julesz,  B.  (1972),  Spatial  Frequency  Masking 
in  Vision  -  Critical  Band  and  Spread  of  Masking,  J.  Opt.  Soc. 

Am. ,  G2 ,  12 . 

27)  Tolhurst,  D.J.  (1975) ,  Sustained  and  Temporal  Channels  in  Human 
Vision,  Vision  Res.,  15,  1151. 

28)  Tyler,  C.W.  (1980) ,  Narrowband  Flicker  Channels  Revealed  by 
Temporal  Frequency  Masking,  Annual  Meeting  Opt.  Soc.  Am., 

Chicago . 

29)  Westheimer,  G.W.  (1979) ,  The  Spatial  Sense  of  the  Eye,  Invest. 
Opthal.  and  Visual  Sci. ,  19. 

30)  Wheeless,  L.L.,  Cohen,  G.H.  and  Boynton,  R.M.  (1967),  Luminance 
as  a  Parameter  of  the  Eye-Movement  Control  System,  J.  Opt.  Soc. 
Am. ,  57 ,  394  . 


276 


31)  Zeevi,  Y-Y.  and  Mangoubi,  S.S.  (1978),  Noise  Suppression  in 
Photoreceptors  and  its  Relevance  to  Incremental  Intensity 
Threshold,  J.  Opt.  Soc.  Am.,  68,  1772. 

32)  Zeevi,  Y.Y.  (180),  Analysis  of  Eye  Movements  in  Target  Tracking 
and  Detection  Tasks,  U.S.  Air  Force,  OSR -SCEE  Report  No. 
F49620-80-C-0038. 

33)  Zeevi,  Y.Y.  and  Gafni,  H.  (1980), Some  aspects  of  Motion  Per¬ 
ception,  Proc.  IEEE  International  Conference  on  Cybernetics, 
Cambridge,  Mass. 


277 


VISUAL  ILLUSIONS 
AND 

VISUAL  SIMULATION 


Duncan  L.  Dieterly 

Chief,  AFHRL  Technology  Office 

NASA-Ames  Research  Center 


Duncan  L.  Dieterly  obtained  his  Ph.D. 
from  the  University  of  Maryland  in 
Industrial/Organizational  Psychology  in 
1975.  His  19  years  of  experience  in  the 
USAF  span  a  spectrum  of  research  and 
management  areas.  His  research  in  human 
factors,  costing,  systems  management  and 
problem  solving  is  well  known. 


VISUAL  ILLUSIONS  AND 
VISUAL  SIMULATION 


imQDUCIXQN 

Visual  illusions  are  occurrences  which  have  fascinated  mankind  for 
centuries.  The  artist  and  magician  have  adjusted  to  the  phenomena  of 
illusions  and  have  capitalized  upon  their  application.  The  artist  to 
perpetuate  an  emotional  feeling  of  depth,  the  magician  to  amaze  with  the 
intent  of  having  the  audience  believe  in  their  powers.  In  both  cases, 
the  phenomena  is  accepted  and  integrated  to  the  advantage  of  the 
craftsman's  objectives.  The  illusion  is  perpetuated  as  reality 
frequently  even  after  it  is  discovered.  Why  then  have  we  who  are 
concerned  with  visual  simulation  not  also  learned  to  use  illusion  rather 
than  avoiding  it  at  all  costs  in  our  attempts  to  mirror  the  reality  of 
the  visual  world? 

The  answer  probably  lies  somewhere  in  the  technicians  dedication  to 
engineering  fidelity  and  lack  of  understanding  the  importance  of 
psychological  fidelity.  Although  psychological  fidelity  is  recognized 
as  a  concept,  it  is  hardly  a  term  found  in  a  contract  specification  for 
a  visual  display  system.  We  must  assume  a  less  parochial  presence  and 
explore  the  psychological  fidelity  of  visual  displays  to  a  greater 
extent.  It  is  time  to  realize,  in  spite  of  all  the  sophisticated 
technology  associated  with  computer  image  generation  (CIC),  we  are  still 
trying  to  capture  a  three-dimensional  world  on  a  two-dimensional 
surface.  We,  therefore,  must  search  for  the  most  functional  ways  to 
expand  the  psychological  fidelity  of  a  CIG  system  as  well  as  routinely 
seeking  the  advanced  technology  of  the  future  which  will  provide  the 
magic  number  of  edges  to  allow  us  the  luxury  of  complete  engineering 
fidelity. 

The  focus  of  this  paper  is  upon  some  of  the  possible  applications 
of  illusions  in  enhancing  visu-1  simulation.  The  objective  is  to 
suggest  ideas  or  directions  which  should  be  explored  in  future 
developments  of  computer  image  generation  systems.  The  ideas  are  not 
completely  crystalized  into  concrete  applications,  but  are  presented  to 
tease,  taunt,  and  intrigue  the  scientist  and  technician  attempting  to 
force  greater  visual  psychological  fidelity  from  existing  systems. 

In  this  paper,  the  technological  limits  of  CIG  simulation  are 
accepted  and  alternative  approaches  to  increase  the  psychological 
fidelity  of  CIG  visual  systems  are  suggested.  An  emphasis  on  the  type 
of  clues  needed  focuses  on  the  application  of  the  knowledge  available 
about  visual  illusions.  Visual  illusions  have  been  employed  for 
centuries  to  obtain  a  psychological  fidelity  as  opposed  to  reproduction 
of  a  scene. 
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ILLUSIONS 


An  illusion  is  an  effect  of  psychological  importance  and  is  only 
labeled  as  an  illusion  by  the  illusionist  who  knows  what  is  actually 
transpiring.  For  the  individual  experiencing  the  illusion  it  is  the 
reality  of  their  perception.  When  an  individual  looks  at  an  ambiguous 
figure  which  reverses  from  an  attractive  woman  to  an  old  hag,  the 
phenomena  may  be  titillating;  but  what  is  actually  occurring?  How  can 
these  lines  be  made  to  convey  only  one  or  the  other  figure?  Admittedly, 
perhaps  no  more  than  an  interesting  parlor  trick,  we  must  recognize  the 
significance  of  this  occurrence  and  be  prepared  for  it.  In  this 
instance  the  illusion  should  alert  us  to  a  possible  error  situation  in  a 
visual  simulation  presentation.  We  must  have  a  series  of  discrete 
dimensions  which  do  not  have  another  figure  or  context  embedded  in  them 
to  adequately  provide  the  required  visual  display.  By  deleting  several 
lines,  either  a  hag  or  woman  remains.  It  is  your  choice,  but  the 
reversal  is  ended  by  reducing  the  ambiguous  lines. 

However,  other  illusions  may  provide  a  simple  way  of  generating 
reality  while  utilizing  their  characteristics.  The  Phi  Phenomena  may  be 
adapted  to  make  moving  lines  or  other  aspects  of  a  display  rather  than 
actually  generating  a  moving  line.  The  after-image  effect  could  be  the 
basis  of  presenting  a  series  of  dashed  figures  with  overlapping  dashes 
that  when  presented  in  a  rapid  sequence  would  appear  as  a  solid  figure. 

I  am  specifically  addressing  the  use  of  illusions  to  a  CIG 
approach,  although  many  of  the  ideas  would  apply  equally  to  any  visual 
simulation  system.  It  would  be  important  for  us  always  to  realize  that 
the  visual  systems  that  are  used  in  all  simulation  capitalize  on  one  of 
man's  greatest  illusions.  That  is  the  illusion  of  movement  by 
synchronized  presentation  of  a  series  of  static  pictures.  We  might  also 
wish  to  consider  alternative  ways  to  further  our  application  rather  than 
forcing  more  edges  for  increased  reality.  What  if  we  speeded  up  the 
presentation  rate  and  introduced  partial  figures  that  overlap?  Would 
they  be  perceived  as  a  total  entity?  If  so,  can  the  speed  be 
manipulated  to  escape  edge  dependency? 

In  addition  to  these  ideas  relative  to  the  technology  associated 
with  a  CIG  system,  should  we  develop  a  research  unit  which  could  use 
animation  and  film  to  establish  the  approaches  that  are  of  interest, 
thereby  saving  the  computer  programming  effort  for  actual  scenes 
generation  until  after  initial  testing  verification  in  a  filmed 
mode.  The  current  complaint  that  CIG  scenes  are  cartoonish  may  be  an 
indication  as  to  where  we  need  to  get  assistance.  Who  has  better  used 
the  concept  of  minimal  cues  to  show  us  reality  than  the  animator?  The 
potential  for  cost  savings  as  well  as  the  introduction  of  a  new  series 
of  technicians  might  increase  the  probability  of  further  success  in 
psychological  fidelity  as  well  as  apparent  engineering  fidelity. 

The  psychologist  and  puzzle  aficionados  have  identified  a  series  of 
two-dimensional  illusions  such  as  the  Pogindorf  Illusion  that  are  simple 
line  drawings.  These  were  generally  considered  as  atypical  phenomena 


which  must  be  explained  away  through  elaborations  of  perceptual 
theories.  Although  a  large  body  of  data  have  been  generated  about 
these,  the  more  interesting  aspect  is  how  far  can  we  take  them  rather 
than  dissipate  them  through  analysis.  What  is  the  outer  limit  of  line 
and  pattern  illusions  and  how  long  does  it  remain  an  illusion  even  after 
the  individual  has  become  educated  about  the  occurrence?  Again  looking 
to  the  art  world,  the  illusions  created  there  are  not  only  lasting,  but 
quite  impressive.  The  use  of  perspective  is  the  basic  ingredient  and 
that  is  but  a  manipulation  of  the  penned  line  to  produce  not  what  the 
artist  sees,  but  what  he  wants  us  to  see.  He  essentially  distorts  his 
learned  perception  back  to  the  actual  perception  of  the  scene.  The 
Mueller-Lyer  illusions,  the  Ponco  illusion,  and  the  vertical-horizontal 
illusion  are  all  examples  of  the  artist's  stock  and  trade. 

Learned  perception  is  the  intellectual  perception  which  over-rides 
the  perceived  stimuli.  This  produces  the  intriguing  phenomena  that  is 
experienced  when  you  turn  a  coin  between  your  fingers  and  perceive  a 
round  object  of  a  specified  value  rather  than  a  set  of  changing  elipses. 
This  is  due  to  a  familiarity  and  learned  perception  which  disregards  the 
actual  information  and  elaborates  upon  it  from  the  individual's  memory 
store.  It  is  the  individual's  learned  perception  which  is  so  critical 
in  understanding  psychological  fidelity.  For  highly  repetitious 
situations  the  perceptual  stimuli  trigger  learned  perceptions  and 
anticipated  actions.  In  the  example  of  the  coin,  the  illusion  is 
unimportant,  but  in  other  situations  it  could  be  dangerous. 

The  more  sophisticated  artistic  displays  of  "anamorphosis"  and 
"trompe  1'  oile"  are  elegant  examples  of  an  extremely  complex 
application  of  perspective,  shadow  and  light,  and  hue  variation. 
Anamorphosis  was  the  production  of  a  distorted  image  which  looked  normal 
when  viewed  from  an  unusual  direction  or  through  a  distorting  mirror. 
Perhaps  only  an  entertainment  for  16th  and  17th  Century  aristocrats,  but 
certainly  a  triumph  in  the  technique  of  two-dimensional  perceptual 
illusions.  "Triumph  L*  Oeil,"  or  the  "deceit  of  the  eye"  is  adequately 
demonstrated  on  the  elegant  ceiling  paintings  of  the  15th  Century  with, 
of  course,  the  Sistine  Chapel  as  the  ultimate  application.  Although  the 
artist  uses  their  skill  to  generate  these  illusions,  they  have  learned 
to  adjust  to  the  individual's  perceptual  limits  and  learned  perception. 

A  cadre  of  artists  have,  therefore,  succeeded  in  grasping  the 
limits  of  human  vision  and  utilized  all  the  illusions  that  they  could 
develop  to  produce  the  reality  that  learned  perception  demands.  As 
Blackmore  states,  "But  even  perception  cannot  proceed  without 
expectations,  built  into  the  brain  in  the  form  of  experiences  with  which 
it  is  familiar."  (p.  91).  It  is  therefore  critical  for  simulation 
technology  to  review  these  artistic  triumphs  in  an  attempt  to  apply  the 
principals  to  CIG  displays.  It  should  also  serve  as  a  warning  in  that 
as  we  seek  reality  through  technological  advancement,  we  may 
inadvertently  introduce  distortion. 
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TYPES  OF  ILLUSIONS 

What  are  the  causes  of  Illusions?  There  would  appear  to  be  four 
categories  of  illusions.  These  are:  (1)  Outside  the  capability  of 
individual,  (2)  on  the  borderline  of  capability,  (3)  contrary  to  learned 
perception,  and  (4)  unusual  or  unknown  phenomena.  In  the  first  case, 
the  individual's  perceptual  limits  are  used  to  deceive  or  confuse  the 
individual.  The  presdigitator's  illusion  of  suspending  a  beautiful 
young  lady  is  based  on  the  low  luminousity  of  the  background  and  the  use 
of  apparently  invisible  wires.  The  reproduction  of  a  video  picture  is 
based  upon  the  limit  of  size  perception  of  the  human  eye  since  it  is 
actually  comprised  of  a  series  of  dots.  By  establishing  the  limits  of 
the  human's  perceptual  system,  the  technicians  may  produce  an  illusion 
of  reality. 

An  example  of  borderline  illusions  are  the  set  of  imbended  figures 
and  after  image  illusions  that  are  well  known.  The  Necker  cube  is  a 
representative  example.  These  illusions  function  at  the  borderline  of 
perception  in  the  sense  they  provide  ambiguous  cues  which  allow  for 
reversed  perceptions  of  the  figure.  They  are  not  designed  for  a  single 
perception  but  multiple  perceptions.  They  occur  at  the  limits  of  our 
perceptual  capacity. 

Those  illusions  which  are  contrary  to  our  learned  perception  are 
the  type  that  artists  use  as  mentioned  earlier  or  the  well-known  Ames 
trapezoidal  room  illusion  with  its  unusual  effects.  The  individual's 
learned  perception  about  rooms  and  the  limitation  of  only  monocular 
viewing  produced  the  perceived  size  effects  obtained.  The  fourth 
category  of  illusion  may  be  only  a  matter  of  experience.  For  example, 
the  illusion  of  a  location  of  a  fish  in  water  caused  by  the  refraction 
of  the  water  may  be  overcome  through  experience,  but  when  first 
encountered,  is  definitely  deceiving.  In  encountering  unknown 
perceptual  patterns,  the  individual  matches  to  their  experience  base. 
An  individual  who  sees  a  rhinoceros  for  the  first  time  may  perceive  it 
as  a  horse  with  a  horn  and  call  it  a  unicorn. 


HE  CESS  ARY  SUES. 

The  perpetual  demand  to  understand  or  determine  the  necessary  cues 
associated  with  a  training  mode  is  still  being  articulated  (Wang,  1980). 
The  cue  issue  is  too  closely  emmeshed  with  a  demand  for  total  reality. 
Individuals  use  cues  and  learn  to  depend  on  a  cue  hierarchy  in  a 
repetitious  environment.  It  allows  them  to  make  choices  when  only  a  few 
cues  are  available  and  provides  them  with  a  priority  system  for 
excluding  less  credible  cues  during  system  overload.  What  these  cues 
are  specifically,  eludes  us  in  the  real  world.  However,  in  the 
laboratory,  research  is  showing  some  interesting  results.  For  example, 
in  a  study  of  pattern  recognition  the  authors  contend  (Hsu  and  Burright, 
1980)  that:  "The  still  very  exciting,  promising  and  eclectically 
supported  idea... that  a  few  (perhaps  about  three)  dimensions  are  basic 
and  critical  to  the  pattern  recognition  problem  for  man  or  machine 


clearly  remains  viable."  (p.  41).  This  finding  supports  the  use  of 
minimal  cues  in  learning  situations  and  emphasizes  the  importance  of 
understanding  these  cues  in  any  given  context. 

Although  major  research  projects  have  attempted  to  determine  cue 
requirements,  none  have  provided  data  that  is  appropriate.  In  the  STRES 
report,  a  major  research  effort,  the  author  could  only  passively  state: 
"However,  available  scientific  and  operational  information  provides 
little  useful  guidance  on  how  to  design  ATO  visual  systems  to  maximize 
training  training  effectiveness."  (Semple,  1981;  p.  13).  The  difficulty 
is  that  we  continue  to  search  for  answers  in  the  available  literature 
when  the  problem  has  never  really  been  addressed  in  the  research  context 
except  in  some  indirect  social  psychological  studies  of  feature 
identification.  The  question  needed  to  be  answered  is  "what  perceptual 
cues  does  an  individual  need  to  respond  correctly?"  Obviously  a 
question  which  becomes  more  complex  as  task  complexity  increases.  Eye 
scanning  studies  of  cockpit  Instruments  during  flight  segments  have 
shown  patterns,  but  not  linked  them  to  performance. 

It  would  appear  reasonable  to  assume  that  individuals  disregard 
extensive  amounts  of  perceptual  information  in  familiar  situations  and 
in  fact  have  a  propensity  for  completing  the  perception  with  anticipated 
and  learned  data.  The  fact  that  you  can  recognize  a  loved  one  with 
almost  no  detailed  perceptual  data  and  while  an  acquaintance  requires 
far  more  data  to  make  the  same  identification  supports  this  contention. 
The  demand  for  a  cue-enriched  environment,  therefore,  diminishes  as 
familiarity  increases.  The  cue-enriched  environment  may  be  reduced  to  a 
basic  minimum  if  the  minimum  were  known.  In  "Nap  of  the  earth"  flying, 
for  example,  users  would  ideally  like  to  have  trees  with  leaves, 
however,  the  cue  necessary  presumably  has  nothing  to  do  with  leaves,  but 
the  color  density  of  the  green  object.  Therefore,  by  taking  a  solid 
green  object  and  varying  intensity  toward  a  speckling  pattern,  the 
necessary  cue  may  be  adequately  simulated.  So,  rather  than  more  detail, 
what  is  required  is  a  perception  changing  from  solid  to  less  dense, 
different-hued  figures  which  simulate  the  rapid  articulation  toward  the 
object. 

Le  Master  and  Longridge  (1978)  found  that  increased  detail  did  not 
aid  in  air-to-ground  target  acquisition.  A  study  reporting  the 
application  of  pseudo  random  noise  codes  to  generate  surface  texturing 
of  infrared  and  low-light-level  television  systems  may  be  generalized  to 
the  Nap-of-the-earth  situation.  (de  Spautz,  et  al,  1980).  They 
developed  a  technique  of  the  use  of  cyclic  codes  for  providing  surface 
detail  that  provides  different  levels  of  detail  for  different 
ranges.  If  this  technique  could  be  linked  to  the  actual  cues  required 
for  a  simulation,  a  powerful  approach  would  be  available  now.  Many 
illusions  are  dependent  upon  low-light  availability  and  limited  texture 
perception. 

As  Semple,  et  al  (1981),  concludes:  "...presumably  there  is  no 
objective  way  of  relating  details  of  the  visual  scene  (e.g.,  field  of 
view,  resolution,  color,  texture,  and  scene  interest)  to  the  process  of 
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human  information  extraction  and  use."  (p.  29).  The  dilemma  only 
becomes  more  complex.  The  psychological  fidelity  of  a  simulation  cannot 
be  considered  directly  unless  the  cue  information  is  available.  General 
knowledge  of  how  individuals  process  cues  does  provide  the  detail 
necessary  to  establish  psychological  fidelity. 

The  phenomena  of  subjective  contour  figures  as  reported  by  Kanizsa 
(1955),  Schumann  (1904)  as  reviewed  by  Coren  (1972)  are  highly  dependent 
on  minimum  cues  and  the  individual's  inclination  to  fill  in  the  missing 
cues  is  a  case  in  point.  The  cue  issue  will  follow  us  into  the  new 
century  but  we  must  attempt  to  mediate  between  total  reality  and  actual 
need  in  terms  of  visual  technology.  If  we  could  establish  the 
appropriate  cues,  it  would  seem  ideal  to  train  in  a  reduced  cue 
environment  and  then  build  towards  a  cue  enriched  environment  rather 
than  the  reverse.  Currently  we  are  trying  to  provide  total  cue 
environments  and  asking  the  trainee  to  magically  pick  out  the  correct 
cues  for  the  task  —  an  interesting  exercise  in  exploration,  but  not 
effect  in  training. 


CONCLUSION 

In  this  brief  presentation  I  have  tried  to  reverse  the  trends  of 
technology  by  suggesting  alternative  ways  of  increasing  psychological 
fidelity.  The  job  in  question  is  accurate  visual  simulation  with  an 
emphasis  on  the  current  merits  of  CIG.  The  possible  use  of  information 
about  illusions  to  reduce  cue  requirements  has  been  suggested.  In 
addition,  the  study  of  minimum  cues  for  success  has  been  indicated  as  a 
possible  method  for  establishing  the  necessary  information  for  training. 
Also  drawing  upon  animation  and  artists  techniques  greater  psychological 
fidelity  may  be  obtained  in  future  CIG  applications.  The  search  for 
increased  fidelity  must  be  considered  as  a  culmination  of  increase 
psychological  and  engineering  fidelity. 


REFERENCES 


Blackmore,  C.:  Mechanlos  of  the  Mind.  Cambridge  University  Press, 
London,  1977. 

Coren,  S.:  "Subjective  Contours  and  Apparent  Depth",  Psychological 
Review.  Vol  79,  No.  4,  pp  359-367,  1972. 

de  Spautz,  J.F.,  Bender,  M.B.,  and  McNamara,  V.M.:  Flight  Training 
Simulator;  Surface  Texture  iia.  Pseudo  Random  Hala.fi.  Cues,  AFHRL 
Text  Report,  AFHRL-TR-80-13,  Brooks  AFB,  Texas,  1980. 

Hsu,  S.  and  Burright,  R.G.;  Comparative  Evaluations  of  RADC/HSU  Texture 
Measurement  System  with  Perceptual  Analysis,  Sequehana  Resources 
&  Environment,  Inc.,  Oct  1980. 

LeMaster,  D.W.  and  Longridge,  T.M.,  Jr.;  Area  of  Interest/Field  of  View 
Research  using  ASPT.  AFHRL  Technical  Report,  AFHRL-TR-78-1 1 ,  (AD 
A055  692) , Williams  AFB,  Arizona,  May  1978 

Semple,  C.A.:  Simulator  Training  Reouirments  and  Effectiveness  Study 
(STRES) :  Executive  Summary.  AFHRL  Technical  Report ,  AFHRL- 

TR-80-63,  Wrignt-Patterson  AFB,  Ohio,  Jan  1981. 

Semple,  C.A.,  Hennessy,  R.T.,  Sanders,  M.S.,  Cross,  B.K.  and  McCaulley, 
M.E. :  Aircrew  Training  Devices;  Fidelity  Features,  AFHRL 

Technical  Report,  AFHRL- TR-80-36 ,  Wright-Patterson  AFB,  Ohio,  Jan 

1981. 


285 


THE  IMPORTANCE  OF  BEING  SQUARE 


Dennis  C.  McCormick 
Applications  and  Training  Analyst 
Singer  Company,  Link  Flight  Simulation  Division 


Dennis  McCormick  is  an  Applications  and  Training  Analyst  in 
the  Advanced  Products  Operation  at  Link  Flight  Simulation 
Division  of  the  Singer  Company,  He  has  been  involved  in  the 
human  factors  of  simulator  visual  systems  during  his  two 
years  with  Link.  Prior  to  this,  he  was  a  pilot  in  the  U.S. 
Navy  flying  both  land  and  carrier  aircraft  and  managing  air 
traffic  control  systems. 

He  holds  a  B.S.  Degree  in  Electrical  Engineering  from  the 
University  of  Idaho  and  an  M.S.  Degree  in  Electrical 
Engineering  from  the  Naval  Postgraduate  School.  He  is 
currently  pursuing  a  degree  in  Industrial  Psychology  at  San 
Jose  State  University. 


286 


THE  IMPORTANCE  OF  BEING  SQUARE 


Abstract 

The  performance  of  a  pilot  controlling  a  simulator  equipped 
with  a  CIG  visual  system  improves  significantly  when  flying 
over  a  data  base  primarily  composed  of  rectangular  surfaces. 
This  is  because  of  an  acuity  to  the  orientation  of 
rectangular  shapes  and  because  the  pilot  makes  valid 
unconscious  perceptual  assumptions  that  the  data  base 
surfaces  are  rectangles.  The  powerful  force  of  erroneous 
perceptual  assumptions  due  to  the  same  mechanism  is 
illustrated  by  a  CIG  illusion  where  the  engine  pylons  of  a 
KC-135  tanker  appear  to  lean  inward  from  their  normal 
position.  The  knowledge  gained  from  determining  the  cause  of 
the  pylon  illusion  and  correcting  the  pylon's  appearance  can 
be  applied  to  data  base  design.  This  will  improve  the 
transfer  of  data  base  spatial  information  to  the  pilot  and 
increase  the  training  effectiveness  of  the  simulator  system. 


Introduction 

Increased  costs  and  safety  restrictions  associated  with  pilot 
training  in  actual  aircraft  has  produced  a  growing  demand  for 
aircraft  simulators  and  trainers  equipped  with  Computer  Image 
Generation  (CIG)  visual  systems.  These  CIG  systems  construct 
a  replica  of  real  world  out-the-window  scenes  through  the  use 
of  visible  surfaces.  Early  CIG  systems  displayed  few  scene 
elements,  but  the  number  of  visible  surfaces  that  the  visual 
systems  can  process  and  display  has  increased  greatly  over 
the  past  few  years.  The  number  of  displayed  surfaces  is 
still  not  sufficient  to  provide  task  essential  spatial 
information  without  careful  data  base  design.  Therefore,  to 
attain  high  levels  of  system  effectiveness,  it  is  important 
to  extract  the  maximum  spatial  information  possible  from  the 
data  base  surfaces.  If  more  were  known  about  the  perceptual 
transfer  of  information,  the  scene  could  be  designed  to  make 
more  efficient  use  of  the  processing  capacity  of  the  CIG 
visual  system.  Improving  the  efficiency  of  information 
transfer  to  the  pilot  and  therefore  increasing  the  effective 
utilization  of  CIG  capacity  is  the  subject  of  this  paper. 

The  surfaces  of  the  visual  world  surrounding  a  pilot  in 
flight  normally  display  a  characteristic  texture.  This,  and 
other  features  aid  the  pilot  to  determine  the  orientation,  or 
slant  and  tilt  of  the  surface.  Perspective  information 
conveyed  by  the  texture  is  a  primary  factor  contributing  to 
the  judgement  of  surface  orientation.  A  surface  that  does 
not  display  texture,  does  not  provide  information  defining 
its  slant  or  tilt  (Carel) .  The  orientation  of  a  terrain 
surface  is  essenial  information  for  the  control  of  low 
altitude  flight.  Thus,  the  surfaces  displayed  by  CIG  visual 
systems  presently  used  for  pilot  training  pose  a  perceptual 
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problem  because  they  do  not  provide  a  perceptually  complete 
texture  to  support  slant  or  tilt  judgements.  Since  the 
surfaces  displayed  by  present  CIG  systems  do  not  possess 
texture,  the  orientation  of  these  surfaces  is,  by  necessity, 
judged  by  the  shape  of  the  surface  as  defined  by  data  base 
edges. 

The  shape  of  CIG  surfaces  have  a  significant  effect  upon  the 
performance  of  a  pilot  during  simulated  lov;  altitude 
maneuvers.  It  has  been  observed  that  pilot  performance 
improves  when  the  CIG  terrain  surface  is  described  by  a 
rectangular  checkerboard  pattern  (Buckland) ,  or  when  scene 
content  rectangularity  is  emphasized  (Bunker).  It  appears 
that  a  significant  factor  causing  the  improved  performance  is 
man's  sensitivity  to  the  orientation  of  rectangles  and  its 
special  case,  the  square.  This  sensitivity  is  often 
attributed  to  the  constant  exposure  of  a  carpentered  world 
(Coren) .  The  vast  amount  of  experience  attained  from 
determining  the  orientation  of  real  world  rectangles  carries 
over  to  the  rectangles  of  the  CIG  world. 

Pilot  judgement  with  respect  to  rectangles  is  good,  however 
there  is  another  significant  factor  which  causes  improved 
pilot  performance  over  a  rectangular  data  base.  The  image 
displayed  by  a  CIG  visual  system  lacks  many  of  the  perceptual 
cues  a  pilot  would  receive  in  the  real  world.  For  example, 
even  though  he  sees  the  image  with  both  eyes,  he  does  not 
receive  real  world  binocular  information  such  as  retinal 
disparity  from  the  CIG  display.  When  the  visual  information 
describing  an  object  in  the  field  of  view  is  not  sufficient 
to  support  an  unambiguous  identification  of  the  object,  the 
observer  makes  perceptual  assumptions  which  provide  the 
necessary  information  to  complete  a  mental  concept  of  the 
object.  These  assumptions  are  based  upon  anticipations  (from 
scene  content,  events,  experience  and  other  factors)  of  what 
should  be  present.  The  mental  concept  of  the  object  is  then 
transformed  in  position,  size,  and  orientation  until  it 
matches  the  sketchy  information  provided  by  the  actual  object 
(Pinker).  The  assumptions  can  be  developed  through  conscious 
thought  such  as  when  a  pilot  attempts  to  recognize  another 
aircraft  after  he  has  detected  it,  or  the  assumptions  can  be 
generated  unconsciously,  when  the  pilot  is  not  aware  of  the 
perceptual  assumptions  being  made.  It  is  an  unconscious 
assumption  that  appears  to  cause  an  interesting  illusion  when 
viewing  a  CIG  data  base  model  of  a  tanker  aircraft  from  the 
refueling  eyepoint  position. 

When  viewing  a  KC-135  tanker  model  from  the  refueling 
position  the  engine  pylons  of  the  tanker,  which  should  appear 
perpendicular  to  the  wings,  appear  to  lean  inward  as  much  as 
30  degrees.  The  apparently  inward  leaning  pylons  can  be  seen 
in  Figure  1.  An  observer's  initial  reaction  to  the  illusion 
is  that  the  CIG  processing  or  the  mapping  of  the  display  is 
in  error,  especially  since  the  amount  of  leaning  increases  as 
the  tanker  is  approached.  Engineering  investigations  did  not 
reveal  any  possible  source  of  the  distortion  in  the  tanker. 
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Then  it  was  suspected  that  the  engine  and  pylon  rotation  was 
caused  by  the  pilot's  visual  system  responding  to  inadequate 
cues.  This  was  confirmed  by  viewing  a  plastic  scale  model  of 
the  aircraft  from  the  refueling  eyepoint.  When  looking  at 
the  engine  pylons  with  both  eyes  they  appeared  to  be  in  their 
correct  orientation.  However,  when  one  eye  was  closed  the 
pylons  leaned  inward  similar  to  the  view  of  the  CIG  tanker. 
In  this  case  the  visual  information  describing  the  scale 
model  aircraft  pylons  was  reduced  by  the  elimination  of 
binocular  cues.  Insufficient  pylon  information  apparently 
causes  the  observer  to  make  unconscious  perceptual 
assumptions  which  produces  the  leaning  pylon  illusion. 

The  major  cue  defining  the  orientation  of  the  pylon  is  its 
shape  which  approximates  a  parallelogram.  Since  there  is 
insufficient  information  to  unambiguously  define  the 
orientation  of  the  pylon,  some  perceptual  assumptions  must  be 
made  by  the  pilot.  It  appears  that  the  observer 
unconsciously  assumes  that  the  shape  of  the  pylon  is  a 
rectangle  and  then  mentally  orients  the  rectangle  in  space  to 
fit  the  image  seen  on  the  CIG  display.  This  perceptual 
process  is  similar  to  that  used  when  viewing  a  perspective 
drawing  of  a  house  (Figure  2) .  The  roof  is  assumed  to  be  a 
rectangle  and  its  parallelogram  shape  defines  its  orientation 
in  3D  space.  (1)  If  the  true  shape  of  the  roof  was  not  a 
rectangle  and  there  was  no  information  to  reveal  its  unusual 
shape,  the  roof  would  appear  slanted  differently  than  it 
actually  is.  This  is  the  situation  that  occurs  with  the 
tanker  pylons. 

For  some  pilots,  the  leaning  pylon  illusion  was  more  than  a 
curiosity.  The  variable  amount  of  leaning,  which  was  a 
function  of  eyepoint  position  relative  to  the  tanker,  also 
had  a  destabilizing  effect  upon  the  apparent  position  of  the 
wing.  For  some  observers,  the  wing  appeared  to  move  slightly 
fore  and  aft,  and  up  and  down.  The  pilot  who  was  susceptable 
to  this  illusion  and  used  the  wing  and  pylons  as  secondary 
cues  to  maintain  his  refueling  position  was  faced  with  the 
difficult  task  of  flying  relative  to  a  rubber  airplane.  To 
increase  the  training  effectiveness  of  the  tanker  model  for 
all  pilots,  it  was  necessary  to  discover  the  reason  for  this 
effect  and  minimize  the  appearance  of  the  leaning  pylon. 

In  order  to  destroy  the  illusion  that  the  pylon  and  engine 
are  leaning,  it  was  necessary  to  provide  additional  spatial 
information  to  control  the  perceptual  assumptions  being  made 
by  the  pilot.  This  requires  an  additional  cue,  or  cues  to 
provide  information  that  the  actual  shape  of  the  pylon  can't 
be  a  rectangle.  It  was  thought  that  if  a  square  was  placed 


(1)  It  has  been  observed  (Stevens),  that  intersecting  lines 
appear  to  cross  at  right  angles  in  space  and  define  a  plane. 
Also,  the  perceptual  default  assumption  of  a  quadrilateral 
figure  is  a  rectangle  (Minskey) . 


on  the  inside  surface  of  the  pylon  it  would  be  perceived 
correctly  which  would  provide  additional  information 
describing  the  orientation  of  the  pylon  surface  and  destroy 
the  illusion.  This  was  done  but  the  resulting  appearance  of 
the  pylon  was  unexpected.  The  pylon  remained  in  its  leaning 
position,  however,  the  square  appeared  to  be  a  vertical  strut 
between  the  v/ing  and  the  pylon.  The  proper  orientation  of 
the  pylon  was  perceived  by  approximating  its  shape  with  a 
series  of  narrow  rectangular  parallelopipeds,  however,  this 
was  unacceptable  because  of  its  unrealistic  appearance  and 
the  high  number  of  surfaces  required  to  model  it.  Eventually 
it  was  discovered  that  the  illusion  can  be  destroyed  by 
adding  the  shadow  of  the  wing  generated  by  an  overhead  sun 
across  the  pylon  and  engine.  This  adds  perceptual 
information  that  makes  it  difficult  to  assume  that  the  pylon 
is  a  rectangle.  The  more  accurate  perception  of  the  pylon's 
orientation  could  be  the  result  of  two  different  strategies. 
Since  the  pylon  can't  be  a  rectangle  due  to  the  information 
provided  by  the  shadow,  then  the  next  reasonable  assumption 
that  could  be  made  is  that  the  pylon  is  perpendicular  to  the 
wing.  Or,  as  stated  by  Lawson  and  Gulick,  "Instead  of 
contours  giving  rise  to  depth  it  is  depth  that  gives  rise  to 
contours."  The  unshadowed  portion  of  the  pylon  provides 
information  that  it  extends  in  depth  foward  of  the  wing  which 
adds  depth  information  and  produces  a  more  accurate 
perception  of  the  pylon's  shape  or  contour. 

The  exact  mechanisms  that  eliminate  the  illsion  is  a  subject 
for  further  study  but  the  effect  on  the  appearance  of  the 
pylon  by  adding  a  simple  shadow  is  dramatic.  A  model  of  the 
tanker  was  modified  such  that  two  of  the  pylons  had  shadows 
and  the  other  two  (on  the  other  side  of  the  aircraft)  did  not 
have  shadows  (Figure  3) .  When  people  see  the  tanker  via  a 
CIG  display,  they  perceive  a  tanker  that  has  normal  pylons  on 
one  side  and  abnormal  pylons  that  are  leaning  inward  on  the 
other  side.  It  is  convincing  evidence  that  the  leaning  pylon 
is  not  a  CIG  error  but  a  human  perceptual  error. 

The  perceptual  assumption  that  causes  the  pylon  illusion  can 
easily  occur  in  other  areas  of  a  CIG  data  base.  If  other 
data  base  features  such  as  terrain  surfaces  are  not 
rectangles  but  could  be  perceptually  assumed  to  be  rectangles 
then  the  perceived  orientation  of  the  surfaces  would  be  in 
error.  This  explains  why  a  data  base  with  an  emphasis  on 
rectangularity  is  more  effective  than  one  that  reproduces  the 
irregular  features  of  the  real  world.  If  the  majority  of 
surfaces  in  a  data  base  are  rectangles  then  most  of  the 
perceptual  assumptions  made  by  the  pilot  are  valid.  When 
viewing  a  data  base  where  the  emphasis  is  placed  on  realism, 
many  erroneous  perceptual  assumptions  can  be  unconsciously 
made  by  the  pilot  which  distorts  the  actual  spatial 
relationships  of  the  data  base.  This  can  occur  even  when  a 
simple  form  of  texture  is  displayed.  Increased  efforts  to 
produce  a  realistic  (and  usually  more  complex)  data  base 
could  foster  more  illusions,  which  reduces  the  perceptual 
fidelity  of  the  data  base.  Without  the  capability  to 
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generate  a  perceptual  real  world  equivalent  surface  texture 
on  all  surfaces,  emphasis  should  be  placed  on  rectangular 
data  base  design  to  insure  perceptual  fidelity. 

Emphasis  on  rectangular  features  and  scene  realism  are  not 
necessarily  incompa table.  It  is  possible  to  design  a  data 
base  that  appears  both  fairly  realistic  and  has  a  high  level 
of  perceptual  fidelity.  This  is  done  by  identifying  task 
relevant  rectangular  features  that  are  present  in  the  real 
world.  These  features  are  then  given  priority  in  the  design, 
generation,  and  display  of  the  data  base.  The  result  is  a 
data  base  that  appears  realistic  because  there  are  few 
artificial  rectangular  shapes.  The  data  base  also  maintains 
a  high  level  of  perceptual  fidelity  because  most  of  the 
visible  features  are  in  fact,  rectangles. 

This  design  approach  has  been  incorporated  in  airport  data 
bases  for  some  commercial  and  government  simulators.  A 
runway  (Figure  4)  appears  realistic  even  though  a  great 
number  of  rectangular  shapes  are  displayed  because 
rectangular  shapes  present  in  the  real  world  runway  are  given 
priority  during  data  base  design.  For  example,  concrete  and 
asphalt  patterns  are  rectangular  and  describe  the  orientation 
of  the  runway  surface  with  a  low  probability  of  generating 
perceptual  errors.  Tire  skid  marks  are  essentially 
rectangular  and  therefore,  they  are  modeled  as  rectangles  to 
avoid  perceptual  errors.  With  the  runway  surface  described 
by  features  that  resist  perceptual  errors  the  efficiency  of 
spatial  information  transfer  to  the  pilot  is  high.  Because 
of  a  greater  quantity  and  quality  of  spatial  information  the 
landing  performance  of  a  pilot  is  improved. 

Summary 

The  primary  purpose  of  flight  simulator  visual  systems  is  to 
convey  to  the  pilot  information  that  is  equivalent  to 
essential  task  relevant  information  presented  by  the  real 
world.  CIG  visual  systems  which  must  provide  this 
information  are  limited  by  the  number  of  surfaces  they  can 
process  and  display.  Therefore,  it  is  important  to  display 
surfaces  that  convey  the  necessary  information  efficiently  to 
maximize  the  effective  use  of  the  CIG  system  processing 
capacity.  This  can  be  accomplished  by  emphasising  the 
presence  of  rectangular  surfaces  within  the  data  base.  It 
should  also  be  recognized  where  non-rectangular ,  four-sided 
surfaces  can't  be  avoided,  additional  surfaces  may  be 
required  to  destroy  an  adverse  illusion.  This  data  base 
design  approach  which  can  be  applied  to  both  present  and 
future  CIG  systems,  will  increase  CIG  system  information 
transfer  efficiency,  perceptual  fidelity,  and  training 
effectiveness. 
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FIGURE  4 
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VISUALLY  INDUCED  SELF-MOTION  SENSATION  ADAPTS 
RAPIDLY  TO  LEFT-RIGHT  REVERSAL  OF  VISION 


ABSTRACT 

After  one  to  three  hours  of  active  movement  while  wearing 
prism  goggles  which  left-right  reverse  the  field  of  view,  10  of 
15  (stationary)  human  subjects  viewing  a  moving  stripe  display 
(back  projected  onto  the  windows  of  a  flight  simulator)  experienced 
a  self-rotation  illusion  (circularvection)  in  the  same  direction 
as  seen  stripe  motion,  rather  than  in  the  opposite  (normal) 
direction,  thus  demonstrating  that  the  central  neural  pathways  which 
process  visual  self-rotation  cues  can  undergo  rapid  adaptive 
modification.  Exposure  to  visual  reversal  apparently  reversed 
the  perceptual  interpretation  of  visual  information  only  from  the 
portion  of  the  subject's  field  of  view  conditioned  by  the  goggles. 
Reversal  of  circularvection  was  not  accompanied  by  a  reversal  of 
the  slow  phase  of  the  vestibulo-ocular  reflex.  Implications  of 
these  findings  will  be  discussed  in  the  context  of  flight  simulation. 

INTRODUCTION 

Both  visual  and  vestibular  motion  cues  are  known  to  contribute 
self-motion  perception  in  a  complementary  fashion1.  In  everyday 
life,  head  rotation  results  in  an  equal  and  oppositely  directed 
angular  motion  of  the  visual  scene  relative  to  the  head.  This 
association  between  normal  active  head  rotation  and  relative  scene 
motion  is  believed  to  account  for  the  phenomenon  of  "circular- 
vection" 2  (CV) :  A  pattern  of  stripes  rotating  around  a  stationary 
observer  soon  elicits  a  compelling  sensation  of  self-rotation  in 
the  opposite  direction3. 

Because  of  the  growing  appreciation  of  the  interplay  between 
visual  and  vestibular  self-motion  cues  in  spatial  orientation  per¬ 
ception  and  control  of  body  movement,  visually  induced  self-motion 
(vection)  has  become  the  focus  of  considerable  research  interest1. 

The  understanding  gained  thus  far1 has  been  exploited  in  applications 
as  diverse  as  neurological  diagnosis  and  flight  simulation.  In  the 
latter  application,  for  example,  cockpit  display  systems  which 
produce  "vection"  are  now  routinely  used  to  alleviate  the  need  for 
large  magnitude  motions  of  the  simulator  cabin.  Many  practical 
applications  in  which  vection  is  exploited  involve  situations  in 
which  the  normal  relationship  between  visual  and  vestibular  cues 
is  disturbed,  the  visual  scene  distorted,  or  the  field  of  view 
limited.  Unless  taken  into  account,  adaptation  to  these  factors 
would  complicate  the  interpretation  of  experiments  on  vection,  and 
should  be  considered  in  evaluating  the  design  of  flight  simulator 
display  systems  and  training  strategies.  Unfortunately,  the  adap¬ 
tive  capabilities  of  the  neural  pathways  which  mediate  the  sensation 
of  vection  have  not  been  systematically  studied.  The  objective  of 
the  present  experiments  was  to  document  the  character  and  time 
course  of  the  vection  adaptation  associated  with  one  particular 
type  of  visual  " rearrangement" " . 


When  human  subjects  wear  optics  which  either  left-right 
("mirror")  reverse  or  invert  (rotate  by  180°)  vision,  the  normal 
relationship  between  left-right  head  rotation  and  relative  visual 
scene  motion  is  reversed.  For  example,  head  rotation  to  the  right 
is  then  accompanied  by  relative  scene  motion  to  the  right  with 
respect  to  the  head,  with  the  result  that  the  seen  world  is  no 
longer  perceived  as  stationary5.  Spatial  orientation  is  severely 
impaired.  However,  after  an  extended  period  of  vision  reversal 
(days  to  weeks),  the  seen  world  appears  more  stable5,  and  sub¬ 
jective  visual  "normalcy"  and  coordinated  movement  are  gradually 
restored6.  Active  movement  by  the  subject  is  thought  to  play  a 
vital  role  in  the  adaptation  process4.  The  slow  phase  component 
of  the  horizontal  vestibulo-ocular  reflex  (VOR) ,  which  contributes 
to  perceptual  stabilization  under  normal  vision,  has  been  shown7 
to  reverse  after  a  week  of  vision  reversal.  The  neural  pathways 
believed  to  mediate  adaptive  changes  in  the  VOR  have  recently  been 
explored  in  animals8.  One  might  also  expect  that  under  visual 
reversal  the  neural  interpretation  of  visual  self-rotation  cues 
would  eventually  adaptively  reverse,  and  be  manifest  perceptually 
in  a  reversal  of  the  CV  phenomenon.  We  have  demonstrated  such  a 
reversal,  accompanied  by  only  a  modest  reduction  in  vestibulo- 
ocular  reflex  gain,  in  10  of  15  subjects  within  a  brief  (1  to  3 
hour)  period  of  exposure.  Reversed  CV  was  only  demonstrated  when 
the  size  of  the  moving  stripe  display  we  used  to  elicit  CV  was 
limited  to  the  field  of.  view  conditioned  by  exposure.  A  prelimin¬ 
ary  report  has  appeared  . 

METHODS 

Three  experiments,  which  differed  somewhat  in  protocol  details, 
were  conducted  using  15  adult  volunteers  with  no  overt  oculomotor 
or  vestibular  disorders.  As  symptoms  of  motion  sickness  often 
occur  under  visual  reversal,  drugs  (scopolamine,  0.5  mg,  or  scopo¬ 
lamine,  0.4  mg/dexedrine ,  5.0  mg)  were  orally  administered  prior 
to  Experiments  I  and  II.  In  all  experiments,  left-right  vision 
reversal  was  achieved  using  the  prism  goggles  shown  in  Figure  1, 
which  permitted  a  binocular  field  of  vision  subtending  approxi¬ 
mately  45  degrees  horizontally  and  28  degrees  vertically1  .  Two 
Dove  prisms,  mounted  base  to  base,  were  held  in  front  of  each  eye 
(with  bases  in  a  saggital  plane)  by  a  plexiglass  frame,  fixed  to 
the  head  by  an  adjustable  band.  A  black  shield  closely  fitted  to 
the  face  and  the  prisms  excluded  all  non-reversed  visual  inform¬ 
ation. 

In  all  experiments,  CV  was  tested  before  and  immediately  after 
a  period  of  exposure  to  re  er  ed  vision  ("preliminary"  and  "final" 
tests).  For  all  subjects  in  Experiment  II,  and  for  one  subject 
in  Experiment  III,  CV  was  walso  tested  at  intervals  during  the 
exposure  period.  Subjects  were  seated  in  the  closed,  motionless 
cabin  of  a  flight  simulator  (Link  GAT-1) .  Equal  width  (6.4°) 
vertical  light  and  dark  stripes,  moving  left  or  right  at  8°/sec 
were  back  projected  onto  the  translucent  front  window,  about  70  cm 
from  the  subject.  During  testing  without  the  goggles  before  and 
after  exposure,  the  shape  of  the  moving  display  could  be  varied 
using  appropriate  masks  applied  to  the  window,  whereas  in  tests 
made  during  exposure,  the  shape  corresponded  to  the  field  of  view 
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of  the  goggles.  Subjects  were  asked  to  verbally  report  the  onset 
and  disappearance  of  any  cv.  Because  of  the  transposition  of  the 
visual  location  of  the  hands  with  goggles  on,  subjects  were  asked 
to  report  their  perceived  direction  of  self-motion  with  respect 
to  the  direction  of  seen  stripe  motion,  with  respect  to  their  left 
and  right  eyes,  or  both.  Motion  reports  made  using  these  two 
different  methods  were  always  consistent. 

The  gain  and  phase  of  the  horizontal  VOR  were  tested  in  the 
dark  before  and  after  exposure  using  sinusoidal  simulator  angular 
velocity  (0.2  Hz,  30°/sec  peak  velocity,  6-8  cycles).  Eye  movement 
was  measured  in  the  dark  using  conventional  DC  electrooculography. 
Subjects  performed  mental  arithmetic  to  maintain  alertness.  (In 
the  case  of  certain  subjects,  we  also  performed  one  or  more  addi¬ 
tional  brief  tests  during  the  preliminary  and  final  test  sessions. 
These  included  acceleration  threshold  measurements,  an  evaluation 
of  gaze  stabilization  during  active  head  movment,  an  oculogyral 
illusion  test,  and  a  pseudorandom  cabin  motion  nulling  test. 

Results  of  these  additional  tests  will  be  reported  elsewhere.)  The 
present  paper  will  focus  on  circularvection  and  VOR  test  results. 

When  not  participating  in  the  brief  pre-exposure  tests,  our 
subjects  explored  their  reversed  visual  environment  by  walking 
through  the  laboratory  buildings  in  the  company  of  an  observer. 

Head  and  body  movements  in  tne  horizontal  plane  were  encouraged. 

When  they  occurred,  motion  sickness  symptoms  were  maintained  at  an 
acceptable  level  by  inter'-,:  „s  of  eye  closure  and/or  head  immobi¬ 
lization. 

RESULTS 

Experiment  I 

In  this  experiment  (5  subjects) ,  we  first  confirmed  that  pre¬ 
exposure  CV  direction  was  opposite  to  seen  stripe  motion  ("normal 
CV")  using  the  63°  (horizontal)  by  28°  rectangular  stripe  display 
shown  in  Figure  2a.  Latency  of  CV  onset  ranged  from  8  to  47  sec. 
Then,  after  the  prism  goggles  were  fitted,  the  subjects  exposed 
themselves  to  vision  reversal  for  180  to  230  minutes. 

After  the  exposure  period,  the  prism  goggles  were  removed, 
and  CV  was  tested  using  a  30°  (horizontal)  by  19°  display,  shown 
in  Figure  2b,  which  stimulated  only  the  field  of  view  previously 
exposed  to  the  goggles.  After  a  latency  of  between  10  and  42 
seconds,  all  five  subjects  reported  an  unequivocal  sensation  of 
motion  in  the  same  direction  as  the  seen  stripe  motion.  Subjects 
usually  likened  the  sensation  to  being  "pulled  along  with  the 
stripes".  We  refer  to  this  perception  as  "reversed  circular¬ 
vection"  (RCV) .  Four  of  the  five  subjects  in  Experiment  I  experi¬ 
enced  compelling  sustained  RCV  until,  some  30  to  60  seconds  later, 
we  suddenly  increased  the  display  size  to  63°  (horizontal)  by  28°. 
Subjects  then  reported  that  RCV  immediately  ceased,  and  after 
several  seconds,  that  normal  (unreversed)  CV  appeared.  Changing 
back  to  the  narrow  field  abolished  normal  CV,  and  RCV  was 
reestablished  after  several  seconds.  This  sequence  could  be 
demonstrated  repeatedly  over  several  minutes.  Active  head  movements 
(eyes  open  in  light)  appeared  to  abolish  RCV.  Our  fifth  subject 
experienced  an  initial  17  second  period  of  compelling  RCV,  followed 
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by  periods  of  normal  CV  and  RCV,  even  though  the  display  area 
remained  narrow. 


Experiment  II 

To  more  systematically  define  the  time  course  of  CV  adaptation, 
seven  additional  subjects  participated  in  a  second  experiment  in 
which  CV  was  systematically  tested  every  30  to  40  minutes  during 
a  190  minute  exposure  period.  To  more  clearly  demonstrate  the 
dependency  of  CV  direction  on  the  region  of  the  visual  field 
stimulated,  CV  was  measured  before  and  after  exposure  using  a 
rectangular  "central  visual  field"  display  (CVF,  29°  horizontal  by 
17°)  shown  in  Figure  3b,  and  also  using  a  "peripheral  visual  field" 
display  (PVF,  180°  horizontal  by  23°,  with  a  dark  mask  of  the  CVF 
display  area),  Figure  3a.  Before  exposure,  the  CVF  test  was 
preceded  by  a  PVF  test.  After  exposure,  two  CVF  tests  (both  direc¬ 
tions  of  stripe  movement)  were  followed  by  a  PVF  test,  and  the  head 
was  immobilized  in  an  effort  to  avoid  readaptation.  Subjects  were 
asked  to  verbally  report  the  first  appearance  of  CV,  its  direction, 
and  their  sensation  of  self-rotation  in  45°  increments.  For  pur¬ 
poses  of  comparison  between  subjects,  we  converted  these  reports 
to  a  four  bin  velocity  scale,  described  in  Table  1.  When  vection 
could  not  be  demonstrated  using  the  display  alone,  we  attempted  to 
trigger  it  with  a  brief  rotation  of  the  simulator  cab,  thus  pro¬ 
viding  a  transient  semicircular  canal  motion  cue  (present  during 
normal  head  rotation,  but  absent  using  conventional  CV  test  pro¬ 
cedures)  .  As  shown  in  Table  1,  four  subjects  demonstrated  RCV. 

The  first  RCV  episode  occurred  after  95  to  190  minutes  of  exposure. 

Experiment  III 

Three  additional  subjects  underwent  an  exposure  period  (which 
ranged  from  150  to  220  minutes) ,  but  used  no  anti-motion  sickness 
drugs.  Symptoms  were  controlled  only  by  eye  closure  and  head 
movement  limitation.  Two  of  the  subjects  were  tested  in  as  in 
Experiment  I,  but  using  only  the  63°  by  28°  front  window  display. 

RCV  was  not  found  after  the  exposure,  perhaps  because  we  failed 
to  test  with  the  narrow  30°  by  19°  display.  The  third  subject 
underwent  a  protocol  similar  to  Experiment  II,  and  demonstrated 
RCV  after  80  minutes  of  exposure  with  goggles  on,  and  with  the  CVF 
display  in  final  testing  with  goggles  removed. 

Vestibulo-Ocular  Reflex  Tests 

VOR  slow  phase  velocity  gain  and  phase  re;  trainer  velocity 
were  measured  in  all  experiments  before  the  first  pre-exposure  CV 
test,  and  after  the  exposure,  just  before  the  goggles  were  removed. 
Data  from  two  subjects  in  Experiment  I  could  not  be  analyzed  due  to 
technical  problems.  In  the  remaining  ten  subjects  in  Experiments 
I  and  II,  VOR  gain  decreased  from  0.72  (S.E.  =  0.35)  to  0.55 
(S.E.  =  0.29)  after  exposure.  This  change  was  significant  (Paired 
Sample  T  Test,  P  <  0.001).  Among  those  seven  subjects  in 
Experiments  I  and  II,  who  experienced  RCV,  again  decreased  from 
0.80  (S.E.  =  0.39)  to  0.64  (S.E.  =  0.31),  also  a  significant  change 
(P  <  0.005).  Slow  phase  eye  velocity  of  the  ten  subjects  lagged 
trainer  velocity  by  180°  (S.E.  =  7°)  prior  to  exposure  and  lagged 
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by  176°  (S.E.  =  11°)  after  exposure.  The  slow  phase  VOR  gain  of 
the  three  Experiment  III  subjects  who  did  not  use  drugs  decreased 
from  0.74  (S.E.  =  0.30)  to  0.47  (S.E.  =  0.17).  We  were  unable  to 
demonstrate  significant  differences  between  the  drug  and  non-drug 
groups  in  terms  of  VOR  gain  decrease.  As  confirmed  by  direct 
inspection  of  the  eye  movement  records,  none  of  our  subjects  showed 
any  evidence  of  reversal  of  the  VOR  slow  phase  component.  (Appar¬ 
ently  systematic  changes  in  the  occurrence  of  the  fast  component 
of  nystagmus  were,  however,  observed.  These  will  be  reported  in 
detail  elsewhere.) 

DISCUSSION 

Our  subjects  showed  approximately  a  20  percent  reduction  in 
VOR  slow  phase  gain  over  their  brief  exposure  period.  This  short 
term  gain  reduction  could  be  due  to  uncontrolled  changes  in  alert¬ 
ness  caused  by  motion  sickness  or  fatigue.  Scopalamine  is  known 
to  depress  oculomotor  responses11,  but  the  gain  reduction  in  our 
drug  and  non-drug  groups  appeared  similar.  Alternatively,  the  gain 
reduction  we  observed  may  represent  the  early  stages  of  the  adaptive 
VOR  change  described  by  Gonshor  and  Jones7.  Their  subjects,  who 
did  not  use  drugs,  showed  similar  gain  changes  during  the  first 
day  of  exposure. 

Our  results  imply  that  vision  reversal  soon  produces  a  corre¬ 
sponding  reversal  of  the  interpretation  of  visual  self-rotation 
cues  from  that  portion  of  the  field  of  view  (referred  to  the  head) 
which  has  been  conditioned  by  the  exposure.  This  change  is  adaptive, 
in  that  it  appears  directed  towards  the  goal  of  veridical  self- 
motion  perception.  The  reversal  of  circularvection  is  presumably 
mediated  by  a  central  mechanism  which  establishes  a  new  association 
between  the  direction  of  motion  of  the  visual  input  relative  to 
the  head  and  self-rotation  direction  information  provided  by  non¬ 
visual  (e.g.  vestibular)  sensory  modalities  during  body  movement. 
Non-visual  self-rotation  information  is  not  available  at  the 
retinal  level.  It  therefore  can  be  argued  that  the  adaptive  pro¬ 
cess  is  central  in  origin,  and  not  a  retinal  phenomenon. 

RCV  could  only  be  demonstrated  when  the  field  of  view  was 
limited  to  or  less  than  the  field  of  view  of  the  goggles.  With  a 
wide  field  stimulus  (as  in  Experiment  I) ,  RCV  appeared  to  be  over¬ 
whelmed  by  normal  vection.  In  Experiment  II,  normal  CV  response 
to  PVF  stripe  motion  was,  if  anything,  enhanced  by  exposure  of  the 
central  field  to  vision  reversal.  Our  findings  suggest  that  visual 
information  from  the  exposed  central  field  of  view  is  transmitted 
centrally  along  pathways  which  are  separate  from  those  carrying 
information  from  the  peripheral  visual  field,  and  that  each  of 
these  pathways  may  be  separately  modified  as  a  result  of  sensory 
experience.  The  occasional  presence  of  alternating  normal  CV  and 
RCV  during  early  per-exposure  tests  indicates  that  RCV  may  not 
develop  gradually.  Instead,  it  is  as  if  a  second,  competitive 
mechanism  develops. 

It  is  important  to  point  out  that  RCV  cannot  simply  be  explain¬ 
ed  as  a  "negative  after-effect"12,  since  RCV  was  demonstrated  with¬ 
out  preceding  unreversed  CV.  Similarly,  RCV  is  distinctly 
different  from  the  brief  episodes  of  "inverted  self-motion  per¬ 
ception"  reported  3  to  occur  during  prolonged  (4  to  12  minutes) 
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optokinetic  stimulation:  RCV  could  be  demonstrated  after  distinctly 
shorter  latencies  (order  of  seconds) ,  with  no  preceding  normal 
vection,  was  sustained  for  periods  greater  than  5  seconds  (usually 
20  seconds  to  more  than  a  minute)  and  could  be  quickly  and  con¬ 
sistently  manipulated  by  altering  the  display  size. 

It  is  not  surprising  that  the  RCV  velocity  magnitudes 
reported  were  modest,  given  the  narrow  field  of  view  of  the  goggles, 
as  normal  CV  can  be  more  effectively  elicited  if  peripheral  retinal 
areas  are  exposed  to  the  moving  display14.  In  this  regard,  we  note 
that  whether  or  not  a  subject  demonstrated  RCV  within  the  allowed 
exposure  period  appeared  to  be  correlated  with  CV  strength  pro¬ 
duced  with  a  narrow  field  stimulus.  In  Experiment  II,  subjects 
B1-B4  showed  strong,  normal  CV  at  least  once,  either  in  CVF  pre¬ 
exposure  tests  or  during  the  first  four  per-exposure  sessions,  and 
they  were  the  only  subjects  in  this  experiment  to  show  RCV.  Test 
scheduling  constraints  limited  exposure  to  190  minutes  in  Experiment 
II.  Had  the  exposure  for  subjects  B5-B7  been  extended,  it  is  con¬ 
ceivable  that  they,  too,  would  have  shown  RCV.  The  exposure 
duration  in  Experiment  I  was  not  so  constrained.  All  five  subjects 
in  Experiment  I  experienced  RCV. 

In  animals,  convergence  of  visual  and  vestibular  head  rotation 
information  is  known  to  occur  in  vestibular  nucleus  neurons15, 
which  are  thought  to  determine  the  slow  phase  velocity  of  vestibular 
and  optokinetic  nystagmus  under  many  conditions.  The  extent  to 
which  vestibular  nucleus  neurons  contribute  to  rotation  perception 
is  unknown,  although  there  appears  to  be  a  close  relationship 
between  the  time  course  of  normal  CV  in  man16  and  the  response 
pattern  of  some  vestibular  nucleus  neurons  in  animals15'1’.  It 
would  be  interesting  to  know  if  a  reversal  in  visual  sensitivity 
of  vestibular  nucleus  neurons  can  be  demonstrated  in  anaimals  after 
several  hours  of  active  exposure  to  vision  reversal. 

CONCLUSIONS 

1.  Reversed  circularvection  (RCV)  was  demonstrated  in  10  of  15 
subjects  after  1  to  4  hours  of  active  head  movement  while 
wearing  left-right  vision  reversing  goggles. 

2.  Exposure  to  left-right  visual  reversal  apparently  reversed  the 
perceptual  interpretation  of  visual  information  only  from  the 
conditioned  portion  of  the  subject's  field  of  view. 

3.  Development  of  RCV  appears  adaptive ,  i.e.  directed  toward  the 
goal  of  veridical  self-motion  perception  and  is  probably  of 
central,  rather  than  retinal  origin. 

4.  RCV  is  quickly  aboloshed  when  active  head  movements  are  made 
eyes  open  in  the  light  with  goggles  off. 

5.  RCV  cannot  be  explained  as  a  negative  after-effect  artifact  or 
as  an  inverted  self-motion  perception  resulting  from  pro¬ 
longed  stimulation.  RCV  was  demonstrated  in  one  subject  with¬ 
out  the  use  of  anti-motion  sickness  drugs. 

6.  Horizontal  VOR  slow  phase  gain  decreased  similarly  in  both  the 
drug  and  non-drug  groups.  No  evidence  of  VOR  slow  phase 
reversal  was  found. 
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Figure  2a: 
Experiment 
the  GAT  -  1 
subject. 


Figure  2b: 
Experi ment 


63  deg.  by  28  deg.  moving  stripe  display  used  in 
I.  Schematic  of  display  as  seen  from  a  position  inside 
simulator  cabin,  above  and  behind  the  head  of  the  seated 


30  deg.  by  19  deg.  moving  stripe  display  used  in 
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Figure  3a:  Central  Visual  Field  (CVF)  moving  stripe  display 
used  in  Experiment  II  (29  deg.  by  17  deg.) 


Figure  3b:  Peripheral  Visual  Field  (PVF)  moving  stripe  display 
used  in  Experiment  II  (180  deg.  by  23  deg.) 
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THE  RELEVANCE  OF  CHANNEL  THEORY 
FOR  THE  DESIGN  OF  SIMULATOR  IMAGERY 


ABSTRACT 

There  is  a  body  of  experimental  evidence  supporting  the  hypothesis  that 
an  early  stage  of  visual  processing  consists  of  analyzing  retinal  image 
information  into  a  number  of  abstract  categories  or  features,  called 
channels.  This  paper  briefly  reviews  the  channel  hypothesis  and  cites 
potential  implications  for  flight  simulator  visual  display  design.  The 
results  of  a  preliminary  study  designed  to  investigate  the  relationships 
between  channel  sensitivity  and  flight  simulator  landing  performance  are 
presented. 


1  .  INTRODUCTION  I 


1.1  THE  CHANNEL  HYPOTHESIS 

It  may  well  be  true  that,  in  most  flying  situations,  the  limits  to  a 
pilot's  performance  are  set  by  his  cognitive  skills  developed  by  prior 
training  and  experience.  Nevertheless,  flying  situations  may  exist  for  which 
his  flying  performance  comes  up  against  the  limited  sensitivity  or 
discrimination  of  his  visual  sensory  system.  For  example,  the  importance  of 
seeing  one's  adversary  before  he  saw  you  was  already  a  byword  among  military 
pilots  when  the  airplane's  major  role  was  as  an  army  scout.  In  more  recent 
times  the  importance  of  visual  sensory  factors  is  implied  by  the  insistence 
that  both  military  and  airline  pilots  should  have  excellent  visual  acuity, 
colour  vision  and  stereoscopic  depth  perception. 

If  we  assume  that,  in  some  situations,  flying  performance  is  limited  by 
visual  sensory  factors,  then  we  might  expect  that  flying  tasks  could  be  found 
for  which  a  pilot's  performance  could  be  predicted  on  the  basis  of  visual 
tests.  However,  this  expectation  has  not  yet  been  fulfilled.  Reports  of 
strong  correlations  between  flying  performance  and  visual  test  results  have  so 
far  been  sparse  (Semple,  Hennessy,  Sanders,  Cross  and  McCauley,  1980). 

During  the  last  10  years  we  have  adopted  the  following  rationale  in 
seeking  visual  tests  that  will  predict  flying  performance.  There  is  a  body  of 
experimental  evidence  supporting  the  hypothesis  that  an  early  stage  of  visual 
processing  consists  of  analyzing  retinal  image  information  into  a  number  of 
abstract  categories  or  features.  In  psychophysical  jargon  this  is 
accomplished  by  passing  the  retinal  image  through  a  few  sets  of 
Information-processing  "channels”  (Braddick,  Campbell  and  Atkinson,  1978; 
Regan,  1981).  The  essential  property  of  these  sets  of  channels  is  that,  to  a 
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first  aoproximation,  they  have  nonoverlaoping  sensitivities  so  that  they 
analyze  the  visual  world  into  independent  categories.  The  individual  channels 
within  a  set  may  have  overlapping  sensitivities.  The  essential  property  of  an 
Individual  channel  within  a  set  is  that  it  operates  independently  of  all  other 
channels.  The  outputs  of  individual  channels  may  be  subtracted  or  otherwise 
reorganized  so  as  to  enhance  discrimination  (e.g.,  as  in  opponent  color 
process).  We  suppose  that,  reorganized  or  not,  the  outputs  of  these  channels 
feed  into  later  stages  of  visual  processing  at  which  cognitive  and  learned 
factors  are  ’mportant,  and  that  to  a  first  approximation,  the  properties  of 
channels  are  not  affected  by  cognitive  factors. 

For  our  present  purpose,  the  significance  of  the  channel  hypothesis  is 
that  channel  processing  wo  Id  be  important  in  visually-guided  flying  tasks, 
and  channel  processing  could  be  largely  described  and  understood  in  ways  that 
are  familiar  in  phvsics  and  electrical  engineering.  Thus,  the  early  "channel" 
stage  of  visual  processing  could  be  more  easily  understood  in  quantitative 
terms  than  the  later  cognitive  stages,  and  the  two  levels  of  processing  could 
be  separated  (Regan,  1981).  Channels  that  have  so  far  been  identified  include 
sets  of  channels  for  colour  (Young,  1802a,  b;  Wright,  1946),  spatial  frequency 
(Campbell  and  Robson,  1968;  Blakemore  and  Campbell,  1969),  stereoscopic 
position  in  depth  (Richards,  1970,  1971),  stereoscopic  motion  in  depth 
(Beverley  and  Regan,  1973,  1975;  Regan,  Beverley  and  Cynader,  1979a,  b), 
changing-size  (Regan  and  Beverley,  1978),  and  sideways  motion  (i.e.,  motion 
parallel  to  the  frontoparallel  plane  (Sekuler,  Pancle  and  Levinson,  1978). 

In  principle,  an  initial  channel  analysis  of  visual  information  could 
offer  a  considerable  advantage  in  acquiring  skills  of  eye-hand  coordination. 
If,  for  example,  learning  how  to  catch  a  ball  can  be  regarded  as  a  process  of 
establishing  one  or  more  "hard  wired"  ways  of  processing  the  outputs  of 
certain  visual  channels,  then  the  skill  would  transfer  from  a  simple  visual 
environment  to  a  complex  visual  environment  because  the  absence  of  overlap 
between  sets  of  channels  and  of  "cross  talk"  between  channels  would  render  the 
initial  channel  analysis  more  or  less  indifferent  to  scene  complexity  (Regan, 
1981;  Regan  and  Beverley,  1980). 

Conversely,  either  too  much  "cross  talk"  between  channels  or  too  much 
overlap  between  sets  of  channels  might  cause  flying  performance  in  complex 
visual  situations  to  fall  below  flying  performance  in  a  simple  visual 
situation.  In  order  to  test  this  suggestion  we  designed  a  tracking  device 
that  measures  a  subject's  ability  to  track  a  test  square  whose  size  changes 
(as  though  moving  in  depth)  while  at  the  same  time  the  square  executes 
side-to-side  movements  (Regan  and  Beverley,  1980).  This  test  assesses  a 
subject's  ability  to  use  his  changing-size  channels  in  conditions  where 
appreciable  cross  talk  is  possible. 

1.2  RELEVANCE  OF  CHANNEL  THEORY  FOR  DESIGN  OF  SIMULATOR  IMAGERY 

If  the  channel  hypothesis  proves  to  be  valid  to  any  substantial  extent, 
there  are  several  possible  implications  for  aviation  research  and  these  are 
discussed  in  detail  elsewhere  (Regan,  1981).  In  brief,  some  flying  tasks 
might  depend  on  only  one,  or  at  any  rate  very  few  sets  of  channels  (Regan, 


1981).  Furthermore,  different  flying  tasks  might  depend  on  different  sets  of 
channels.  If  this  is  the  case,  then  visual  test  results  on  one  set  of 
channels  might  predict  performance  in  one  flying  task,  while  test  results  on 
other  sets  of  channel  might  predict  performance  in  a  second  flying  task,  and 
this  is  currently  under  experimental  investigation. 

If  the  only  visual  information  available  for  later  cognitive  processing 
is  the  outputs  of  the  various  sets  of  channels,  then  the  task  of  simulator 
imagery  reduces  to  effectively  driving  the  relevant  sets  of  channels.  If  a 
particular  set  of  channels  is  not  involved  in  the  visual  tasks  being  learned, 
then  that  set  of  channels  need  not  be  driven  by  a  visual  display  used  in 
training  for  this  task.  Finally,  the  minimum  dynamic  and  spatial  (e.g. , 
resolution)  performance  of  a  visual  display  could  be  defined  by  reference  to 
the  dynamic  and  spatial  frequency  properties  of  the  channels  involved  in  the 
visual  tasks  to  be  learned.  It  would  be  unnecessary  for  the  visual  display  to 
have  a  wider  temporal  or  spatial  frequency  bandwidth  than  the  widest  bandwidth 
of  the  relevant  channels. 

All  this  depends  on  the  validity  of  the  channel  hypothesis,  but  the 
channel  hypothesis  as  set  out  in  Section  1.1  above  is  subject  to  experimental 
test  and  experimental  refutation.  In  particular,  the  following  points  are 
open  to  experimental  refutation:  (a)  non-overlapping  sensitivities  of  sets  of 
channels;  (b)  independent  operation  of  different  channels;  (c)  lack  of 
cognitive  effect  upon  channel  properties. 


2  .  BACKGROUND 


2.1  STEREOSCOPIC  PERCEPTION  OF  POSITION- IN-DEPTH  AND  OF  MOT ION- IN-DEPTH 

The  designers  of  flight  simulators  have  generally  reduced  costs  by  not 
including  a  stereo  visual  display  on  the  grounds  that,  since  classical  stereo 
vision  is  almost  ineffective  at  distances  over  10  metres  or  so,  it  is  unlikely 
to  be  important  in  flying.  The  purpose  of  Section  2.1  is  to  suggest  that, 
even  when  cost  and  complexity  prevent  the  inclusion  of  a  stereo  display,  the 
simulator  designer  should  note  that  the  human  visual  pathway  contains  a  stereo 
motion  system  that  may  be  important  out  to  longer  viewing  distances  than 
classical  stereo  vision.  This  stereo  motion  system  might  be  involved  not  only 
in  landing  helicopters  on  oil  rigs  or  ships  and  in  nap-of-the-earth 
operations,  but  also  in  some  tasks  with  fixed-wing  aircraft.  It  is  therefore 
possible  that  unexpected  transfer  of  training  difficulties  might  be 
experienced  as  a  result  of  omitting  a  stereo  display. 

Psychophysical  evidence  that  motion  in  depth  and  position  in  depth  are 
processed  separately. 

Recent  evidence  supports  the  idea  that  there  are  separate  stereoscopic 
channels  for  position  in  depth  and  for  motion  in  depth.  Beverley  and  Regan 
M9731  pointed  out  that  the  relative  velocities  of  the  left  and  right  retinal 


images  provide  a  precise  cue  to  the  direction  of  motion  in  depth  (Figure  1). 
Their  evidence  that  the  human  visual  pathway  uses  this  cue  was  obtained 
psychophysically.  They  found  that,  after  inspecting  an  image  oscillating  in 
depth,  visual  sensitivity  to  motion  in  depth  was  depressed.  The  important 
point  was  that  only  a  narrow  range  of  directions  of  motion  in  depth  was 
affected.  They  proposed  that  there  are  several  stereoscopic  motion-in-depth 
channels,  tuned  to  four  different  directions  in  depth.  Figure  2B  shows  the 
directional  tuning  of  these  channels^for  one  human  subject. 

Richards  and  Regan  (1973)  reported  a  different  sort  of  evidence  that 
binocular  disparity  is  processed  separately  from  the  relative  velocities  of 
the  left  and  right  retinal  images.  They  found  subjects  whose  visual  field 
contained  regions  that  were  "blind"  to  disparity,  yet  were  sensitive  to  motion 
in  depth,  while  other  regions  of  the  visual  field  were  blind  to  motion  in 
depth,  yet  sensitive  to  disparity. 

In  brief,  the  newly-discovered  stereo  system  for  motion  is  quite  distinct 
and  different  from  the  classical  stereo  system  for  position  in  depth.  Testing 
a  subject's  classical  stereo  vision  does  not  necessarily  tell  us  about  his 
stereo  vision  for  motion.  In  view  of  the  failure  of  classical  stereo  vision 
at  distances  over  10-20  metres  it  might  seem  surprising  that  the  ratio  between 
stereo  and  monocular  sensitivities  for  motion  in  depth  does  not  depend  on 
viewing  distance. 

A  further  distinction  between  the  stereo  motion  channels  and  the 
classical  stereo  position  channels  is  that  their  dynamics  are  very  different 
(Regan  and  Beverley,  1963a).  Figure  3  illustrates  this  point;  Panel  A  shows 
how  visual  sensitivity  to  sideways  oscillations  depends  on  oscillation 
frequency.  Visual  sensitivity  is  best  i.e.,  threshold  is  lowest)  for 
oscillation  frequencies  between  about  1  and  5  Hz,  but  oscillations  can  still 
be  seen  up  to  25  Hz  at  least.  In  contrast,  panel  B  is  a  similar  plot  for  the 
stereo  motion  channels.  Visual  sensitivity  for  oscillations  in  depth  falls 
off  from  the  lowest  frequency  tested  (about  0.8  Hz),  and  oscillations  in  depth 
cannot  be  seen  at  all  above  about  3-5  Hz.  Putting  this  another  way,  the 
stereo  motion  channels  are  far  more  sluggish  than  the  sideways  motion 
channels.* 


*  The  stereo  motion  data  of  panel  B  was  obtained  as  follows.  The  subject's 
left  eye  viewed  a  bar  or  dot  pattern  oscillating  from  side  to  side  at  a 
frequency  F  Hz  while  the  right  eye  viewed  a  similar  pattern  oscillating  at 
(F  +AF)  Hz.  In  binocular  fusion  the  target  appeared  to  change  periodically 
in  depth  at  a  frequencyAF  Hz,  but  though  this  frequencyAF  was  available  in 
binocular  view,  it  was  not  available  to  either  left  or  right  eye  alone.  Panel 
B  shows  how  visual  sensitivity  to  theAF  Hz  movements  in  depth  were  affected 
by  the  value  of  AF  (Regan  and  Beverley,  1973a). 


The  ability  to  discriminate  between  different  directions  of  motion  in  depth 


Figure  2A  shows  a  subject's  ability  to  discriminate  between  different 
directions  of  motion  in  depth  {Beverley  and  Regan,  1975).  This  ability  is 
remarkably  acute,  a  difference  of  as  little  as  0.1°  to  0.2°  in  direction 
being  detectable.  Figure  2A  might  help  to  explain  the  well -recognized  ability 
of  tennis  and  baseball  players,  automobile  drivers  and  pilots  to  make  very 
acute  judgements  of  the  direction  of  motion  in  depth.  This  high  acuity  cannot 
be  entirely  explained  in  terms  of  the  directionally-selective  channels  of 
Figure  28,  since  they  would  predict  an  acuity  of  only  about  1°  to  2°. 

Beverley  and  Regan  proposed,  therefore,  that  the  outputs  of  the  channels  of 
Figure  2A  are  subtracted  at  a  later  stage  so  as  to  give  the  acute 
discrimination  of  Figure  2A,  much  as  the  outputs  of  red,  green  and  blue  cones 
pass  through  an  opponent  colour  process  and  thus  mediate  our  acute  colour 
discrimination. 

Brain  cells  sharply  tuned  to  the  direction  of  motion  in  depth  and  brain 
cells  sharply  tuned  to  position  in  depth 

Recording  from  areas  18  and  from  the  17/18  border  in  cat  visual  cortex, 
Cynader  and  Regan  (1978,  1981)  found  single  neurons  that  were  selectively 
sensitive  to  the  ratio  between  left  and  right  retinal  image  velocities.  In 
other  words,  these  neurons  were  tuned  to  the  direction  of  motion  in  depth.* 

All  the  psychophysical  motion-in-depth  channels  in  man  shown  in  Figure  2  had 
their  equivalent  neurons  in  cat  visual  cortex.  Some  of  the  most  sharply-tuned 
neurons  fired  best  when  the  retinal  images  moved  in  opposite  directions,  and 
thus  responded  to  a  range  of  directions  spanning  no  more  than  1°  to  2°. 

Many  of  these  neurons  maintained  their  directional  tuning  over  at  least  a 
four-fold  range  of  speeds  (Figure  4A).  A  second  class  of  neurons  fired  best 
for  trajectories  that  missed  the  head.  These  were  tuned  to  a  broader  range  of 
directions  than  the  "hitting  the  head"  class  of  neurons.  Similar  "motion  in 
depth"  neurons  have  recently  been  found  in  visual  cortex  of  unanaesthetized, 
behaving  monkey  (Poggio  and  Talbot,  1981),  so  that  it  seems  likely  that 
equivalent  neurons  exist  in  human  visual  cortex. 

Cynader  and  Regan  (19781  also  found  a  class  of  neuron  that  had  been 
previously  described,  these  neurons  showed  strong  interocular  facilitation 
Tup  to  100-fold)  when  motion  was  accurately  parallel  to  the  frontoparallel 
plane  (Figure  4B).  This  class  of  neuron  also  can  be  regarded  as  being  very 
selective  to  the  direction  of  motion  in  depth. 


*  Note  that  these  motion-in-depth  neurons  cannot  be  described  as  being 
selectively  sensitive  to  the  rate  of  change  of  disparity,  i.e.,  to  the 
difference  between  the  left  and  right  image  velocities  since  many  directions 
of  motion  correspond  to  a  single  rate  of  change  of  disparity  (Regan  and 
Cynader,  1981). 
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This  third  class  of  neuron  was  comparatively  sharply  tuned  to  disparity, 
i.e.,  to  position  in  depth.  In  this  they  differed  from  many  neurons  in  the 
first  two  classes,  some  of  which  maintained  their  tuning  to  motion  in  depth 
over  a  range  of  disparities  as  large  as  12°.  Other  neurons  of  this  type 
systematically  changed  their  tuning  as  a  function  of  disparity,  for  example, 
favouring  motion  towards  the  frontoparallel  plane  both  for  objects  located 
nearer  than  the  frontoparallel  plane  and  for  objects  located  beyond  the 
frontoparallel  plane. 

Evoked  potentials 

Evoked  potential  studies  bridging  the  inter-species  gap  between  our 
psychophysical  studies  in  man  and  animal  single-unit  experiments  were  carried 
out  before  these  other  studies  were  started.  Regan  and  Spekreijse  (1970)  used 
Julesz  patterns  to  obtain  evoked  potentials  to  stereoscopic  motion  in  depth. 
Regan  and  Beverley  (1973b)  later  found  that  different  evoked  potentials  were 
produced  by  different  directions  of  motion  in  depth,  anticipating  the 
conclusions  of  subsequent  psychophysical  and  single-unit  studies. 

2.2  VISUAL  PROCESSING  OF  CHANGING-SIZE  AND  SIDEWAYS  MOTION 

Psychophysical  evidence  that  chanqinq-size  and  sideways  motion  are  processed 
separately 


Since  the  channels  described  above  are  sensitive  to  the  relative  velocity 
of  the  retinal  images,  they  can  only  be  driven  monocularly.  A  second  stimulus 
for  motion  in  depth  is,  however,  effective  monocularly  as  well  as 
binocularly.  This  is  the  stimulus  of  changing  size. 

Figure  5  shows  experimental  evidence  that  the  processing  of  changing-size 
information  cannot  be  explained  in  terms  of  responses  to  motion.  Subjects 
viewed  a  pair  of  bright  squares  on  a  dimmer  background.  We  used  the  following 
two  stimulus  osci nations:  inphase  oscillations,  in  which  opposite  edges  of  a 
square  moved  in  the  same  direction  at  any  instant;  antiphase  oscillations,  in 
which  opposite  edges  of  a  square  moved  in  opposite  directions  at  any  instant. 
Subjects  first  measured  the  smallest  oscillation  amplitude  that  could  just  be 
detected  for  an  inphase  test  oscillation  and  for  an  antiphase  test 
oscillation.  Then  they  inspected  a  strong  antiphase  oscillation  for  20  min 
and  measured  the  two  thresholds  again.  The  antiphase  threshold  was  much 
elevated,  but  the  inphase  threshold  was  comparatively  little  affected.  When 
the  experiment  was  repeated  with  an  inphase  adapting  oscillation,  threshold 
elevations  were  small  for  both  test  stimuli  (Regan  and  Beverley,  1978). 

This  differential  effect  was  still  observed  when  the  adapting  stimulus 
was  a  bright  square  on  a  dark  ground  and  the  test  stimulus  was  a  dark  square 
on  a  bright  ground. 
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A  weakness  of  these  early  experiments  is  that  they  were  limited  to  two 
directions  of  motion,  namely  motion  parallel  to  the  frontoparallel  plane 
(i.e.,  inphase  oscillations),  and  motion  along  a  line  through  the  eye  (i.e., 
antiphase  oscillations).  A  more  recent  experiment  extended  the  conclusions  to 
a  range  of  adapting  trajectories.  The  rationale  of  this  later  experiment  is 
illustrated  in  Figure  6A.  On  separate  days,  subjects  adapted  to  11  different 
trajectories.  Each  adapting  oscillation  had  the  same  velocity  along  a  line 
through  the  eye,  i.e.,  the  same  antiphase  component,  but  each  adapting 
oscillation  had  a  different  velocity  parallel  to  the  frontoparallel  plane 
(i.e.,  a  different  inphase  component  of  oscillation).  If  the  changing-size 
channel  responded  to  the  antiphase  component  and  was  unaffected  by  the  inphase 
component,  then  all  11  adapting  oscillations  should  produce  the  same  threshold 
elevation  for  the  antiphase  test  stimulus. 

Figure  6B  shows  that  this  prediction  was  upheld  to  an  accuracy  of  about 
+30%  over  a  very  wide  span  of  trajectories.  (The  abscissa  in  Figure  6B  plots 
Tnphase  components  ranging  from  zero  to  +8  times  tne  amplitude  of  the 
antiphase  component.)  Further  evidence  suggested  that,  in  everyday  vision, 
the  prediction  probably  holds  to  considerably  better  than  +30%  due  to  the 
effect  of  eye  movement.  The  experiment  of  Figure  6B  was  repeated  with  a 
rapid  "jitter"  oscillation  added  to  the  adapting  stimulus.  As  shown  in  Figure 
6C,  the  effect  of  this  "jitter"  was  to  reduce  the  influence  of  an  inphase 
component  upon  the  antiphase  threshold  elevation  so  that  the  influence  was 
less  than  measurement  errors  (+5%)  (Regan  and  Beverley,  1980).  "Jitter"  was 
most  effective  when  it  approximated  the  natural  retinal  image  "jitter"  that 
occurs  in  everyday  life  when  the  head  is  not  artificially  supported. 

In  order  to  explain  these  findings  we  proposed  that  the  visual  system 
contain  a  set  of  channels  that  operate  as  illustrated  in  Figure  7.  The 
accuracy  with  which  the  computations  (x-y)  and  (x+y)  are  carried  out  (better 
than  5%)  may  seem  astonishing,  especially  since  the  accuracy  is  almost 
independent  of  the  absolute  values  of  x  and  y.  It  may  seem  less  surprising 
when  we  reflect  how  casually  we  rely  on  such  computations  when  judging  the 
motion  of  speeding  cars  on  busy  roads. 

In  brief,  these  findings  suggest  that  the  set  of  changing-size  channels 
allow  the  visual  system  to  directly  compute  the  1 ine-of-sight  component  of  an 
object's  velocity,  that  is  the  velocity  component  along  a  line  through  the  eye 
n/|_os  'in  Figure  8)  even  when  both  object  and  head  are  moving  along  arbitrary 
trajectories. 

Brain  neurons  sensitive  to  changing  size 

All  101  neurons  studied  by  Regan  and  Cynader  (1979)  in  area  18  of  cat 
visual  cortex  responded  to  changing-size.  However,  control  experiments 
eliminated  most  of  these  neurons  by  showing  that  they  were  responding  to 
changes  in  light  flux  or  to  motion  of  a  single  edge  rather  than  to  changing 
size.  Nevertheless,  a  very  few  neurons  in  area  18  did  satisfy  the  criteria 
for  changing-size  responses.  In  addition,  there  was  evidence  that  a 


315 


r,_*r 


considerable  proportion  of  neurons  slightly,  but  systematically  emphasized 
changing-size  even  though  no  systematic  preference  was  evident  in  simple  spike 
counts. 

2.3  TWO  STIMULI  FOR  MOTION- IN-DEPTH:  COMPARATIVE  EFFECTIVENESS  OF 
CHANGING-DISPARITY  AND  CHANGING-SIZE 

Figure  9  illustrates  that  when  an  object  moves  directly  toward  the  head, 
the  two  retinal  images  move  away  from  each  other  and  also  grow  larger.  As 
discussed  above,  either  of  these  two  visual  inputs  by  itself  is  capable  of 
causing  a  sensation  of  motion  in  depth.  In  the  real  visual  world  the 
situation  is  almost  always  as  shown  in  Figure  9,  but  in  order  to  measure  the 
relative  effectiveness  of  the  two  inputs,  we  created  artificial  stimuli  in 
which  the  two  inputs  of  Figure  9  were  antagonistic  (for  example,  as  an  object 
came  closer  it  grew  smaller). 

Subjects  viewed  a  square  whose  size  changed  at  a  fixed  rate,  so  that  the 
square  appeared  to  move  in  depth.  The  rate  of  change  of  binocular  disparity 
was  then  adjusted  by  the  subject  so  as  to  exactly  cancel  the  impression  of 
motion  in  depth  produced  by  the  fixed  rate  of  change  of  size  (Regan  and 
Beverley,  1979a).  It  is  possible  to  use  Figure  10  to  read  off  a  subject's 
relative  sensitivity  to  monocular  and  stereoscopic  stimuli  for  motion  in 
depth,  given  the  inspection  duration  and  speed. 

In  a  number  of  flight  tests,  the  focus  of  discussion  has  been  whether  the 
absence  of  binocular  vision  affects  pilot  performance.  Three  flight  studies 
have  found  that  the  landing  performance  of  pilots  in  daylight  was  not  degraded 
by  occluding  one  eye  during  landing  (Pfaffman,  1948;  Lewis,  Blakeley,  Swaroop, 
Masters  and  McMurty,  1973;  Grosslight,  Fletcher,  Masterton  and  Hagen,  1978) 
and  one  study  reported  that  landing  performance  could  even  be  improved  by  loss 
of  binocular  vision  (Lewis  and  Kriers,  1969)  though  this  finding  has  been 
challenged  by  researchers  who,  nevertheless,  found  that  monocular  performance 
was  no  worse  than  binocular  performance  (Grosslight,  Fletcher,  Masterton  and 
Hagen,  1978).  In  one  of  these  studies  the  suspicion  that  pilots  might  try 
harder  when  one  eye  was  occluded  was  addressed  by  repeating  the  flight  tests 
with  low-time  pilots  rather  than  experienced  military  pilots.  Even  though  the 
low-time  pilots  were  presumably  stretched  when  landing  normally  with  full 
binocular  vision,  monocular  occlusions  produced  no  reduction  in  landing 
performance  (Lewis  and  Kriers,  1969). 

The  results  of  these  flight  tests  are  consistent  with  the  notion  that 
static  binocular  depth  perception  is  unimportant  at  a  range  in  excess  of  about 
10  yards  when  other  depth  cues  are  available  (Howard  and  Templeton,  1966). 

However,  judgements  of  motion-in-depth  are  a  quite  different  matter  from 
judgements  of  position  in  depth.  Our  laboratory  experiments  suggest  that  the 
flight  tests  discussed  above  may  have  confounded  several  factors,  so  that  the 
results  of  these  flight  tests  may  apply  to  only  a  narrow  range  of  visual 
situations.  Consider  the  perception  of  the  motion  in  depth  of  ground  objects 
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during  landing.  If  binocular  vision  is  lost,  the  consequence  for 

mot  ion- in-depth  sensitivity  would  depend  on  the  following  five  factors  (Regan 

and  Beverley,  1979a). 


(1)  Inspection  duration.  A  loss  of  binocular  vision  would  be  less 
important  for  short  inspection  times. 

< 2 )  The  target's  velocity  in  depth.  Visual  sensitivity  to  low 
velocities  would  be  less  affected  by  losing  binocular  vision. 

(3)  The  absolute  width  of  the  object.  Visual  sensitivity  to  the 
motion-in-depth  of  wider  objects  would  be  less  affected  by  the  loss  of 
binocular  vision.  (Except  at  very  short  distances,  this  would  be  independent 
of  distance;  see  Regan  and  Beverley,  1979a,  Appendix). 

(41  The  difference  between  binocular  and  monocular  threshold  for 
mot  ion- in-depth  sensation  produced  by  changing-size  stimulation.  For  our 
subjects  and  stimulus  conditions,  binocular  thresholds  were  up  to  about  1.4 
times  lower  than  monocular  thresholds  (depending  on  the  individual  subject). 

(5)  Individual  differences.  These  are  large.  Even  over  our  small 
sample  of  five  subjects  we  found  a  greater  than  80:1  intersubject  difference 
in  the  relative  effectiveness  of  changing-size  and  changing-disparity  in 
producing  a  sensation  of  motion-in-depth. 

The  following  calculation  is  intended  to  illustrate  this  point.  An 
airplane  travelling  at  140  mph  is  supposed  to  be  2000  ft  from  a  runway 
measuring  100  ft  wide.  To  anticipate,  if  the  pilot  (subject  T.W.  in  Figure 
101  looked  at  the  runway  for  1.0  sec,  the  monocularly-avai lable  cue  of 
changing-size  would  be  76  times  more  effective  than  changing-disparity  as  a 
stimulus  for  motion-in-depth. 

At  2000  ft  the  angular  width  of  the  runway  is  2.72°.  After  1  sec 
approaching  at  140  mph,  the  angular  width. increases  to  3.02°.  Hence 
the  mean  rate  of  change  of  angular  width  9$  =  0.295°/sec  =  17.7  min 
arc  sec  (i.e. ,  8.86  min  arc  sec  for  each  edge). 

For  subject  T.W.  the  rate  of  change  of  disparity  (9g)  equivalent  to 
9$  =  17.7  min  arc  sec  for  an  inspection. time  of  1  sec  is  9g  = 

1.4  min  arc  sec  (from  Figure  10),  hence  Og/dj  =  0.16. 

From  the  geometrical  considerations  outlined  in  Appendix  1,  we  would 
expect  that  6g/9$  =  I/S.  In  this  case  S  =  100  ft  and  I  =  2.5  in.,  as 
the  geometrical y  predicted  ratio  9g/9$  is  0.0021. 

Hence,  allowing  for  geometric  factors,  changing-size  would  be  76 
times  more  effective  than  changing-disparity  as  a  stimulus  for  motion  in 
depth,  for  subject  T.  W. 
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2.4  DYNAMIC  PROPERTIES  OF  THE  CHANGING-SIZE  AND  STEREOSCOPIC  MOTION  CHANNELS 


Figure  11  plots  thresholds  for  just-detectable  motion  in  depth  produced 
by  two  different  visual  inputs,  namely  changing-size  (continuous  line),  and 
changing-disparity  (bold  broken  line).  The  two  plots  are  clearly  different. 
In  particular,  changing-size  grows  relatively  less  effective  at  oscillation 
frequencies  below  about  1  Hz. 

Figure  11  also  plots  thresholds  for  two  different  sensations  produced  by 
a  single  input.  The  input  is  changing-size,  and  the  two  sensations  are 
changing-size  (fine  dotted  line),  and  motion  in  depth  (continuous  line).  The 
chief  difference  between  the  two  curves  is  that  changing-size  is  ineffective 
as  a  stimulus  for  motion  in  depth  for  a  range  of  frequencies  above  about  3  Hz 
although  at  these  high  frequencies  it  is  still  effective  in  producing  a 
sensation  of  changing-size. 

2.5  MOTION  OF  THE  SUBJECT  THROUGH  A  FIXED  EXTERNAL  WORLD 


Gibson  (1950)  discussed  how  a  visual  flow  pattern  can  produce  the 
impression  that  one  is  moving  through  the  external  world  towards  the  focus  of 
the  flow  pattern.  He  pointed  out  that  this  is  an  important  visual  cue  for 
automobile  drivers  and  pilots. 

Testing  the  suggestion  that  changing-size  channels  might  be  involved  in 
visual  sensitivity  to  flow  patterns,  Regan  and  Beverley  (1979b)  measured 
elevations  of  changing-size  thresholds  caused  by  adapting  to  a  flow  pattern. 
Their  finding  that  changing-size  thresholds  were  indeed  elevated,  but  only  for 
locations  near  the  focus  of  the  flow  pattern,  gave  some  support  to  those 
suggestions  (Regan,  Beverley  and  Cynader,  1979b).  This  experiment  indicates 
that  visual  sensitivity  to  expanding  flow  patterns  is  not  necessarily  a 
large-field  phenomenon.  A  small-field  mechanism  may  also  be  involved,  though 
we  should  emphasize  that  our  proposal  is  intended  only  as  a  partial 
explanation  for  visual  localization  of  a  flow  pattern's  focus. 

2.6  SPATIAL  FREQUENCY  CHANNELS 


Mathematical  discoveries  published  by  Fourier  as  long  ago  as  1807 
(Herivel,  1975)  led  to  the  realization  that  any  picture  or  scene  can  be  broken 
down  into  many  superposed  sinewave  gratings  and  conversely  that,  by 
superposing  the  appropriate  sinewave  gratings,  any  picture  can  be 
synthesized.  By  the  1950 ' s ,  this  fact  was  the  basis  of  a  standard  technique 
for  assessing  the  performance  of  lenses  for  aerial  photography  (Hopkins,  1957). 

It  was  a  considerable  further  step  beyond  the  mathematical  facts  to 
propose  that  the  human  visual  system  analyzes  the  retinal  image  into  spatial 
frequency  bands.  Campbell  and  Robson  (1968)  proposed  that  the  human  visual 
system  contains  "functionally  separate  mechanisms  .  .  .  each  responding 
maximally  at  some  particular  spatial  frequency  and  hardly  at  all  at 
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frequencies  differing  by  a  factor  of  two.  The  frequency  selectivity  of  these 
mechanisms  must  be  determined  by  integrative  processes  in  the  nervous  system, 
and  they  appear  to  a  first  approximation  to  operate  linearly." 

It  is  not  yet  clear  what  these  spatial  frequency  channels  correspond  to 
physiologically.  Some  neurophysiologists  have  found  neurons  in  the  visual 
pathway  of  animals  that  have  wave-like  receptive  fields  so  that  they  respond 
best  to  sinewave  grating  stimuli,  and  could  act  as  physical  Fourier  analyzers 
(Pollen  and  Feldon,  1979).  Other  vision  scientists  favour  quite  different 
explanations,  for  example  that  every  location  on  the  retina  has  only  four 
classes  of  receptive  field  size,  and  that  the  action  of  four  classes  is 
equivalent  to  analyzing  the  retinal  image  into  four  spatial  frequency  bands 
(Wilson  and  Bergen,  1979). 

But  whatever  the  physiological  basis  of  spatial  frequency  channels  may 
be,  for  our  present  purpose  the  main  point  is  that  human  visual  sensitivity 
for  fine  detail  (i.e.  high  spatial  frequencies)  does  not  necessarily  tell  us 
about  human  vision  for  coarse  and  intermediate  detail  (i.e.  low  and 
intermediate  spatial  frequencies).  A  clear  illustration  of  this  point  is 
given  by  the  finding  that  many  multiple  sclerosis  patients  with  normal  Snellen 
acuity  have  severely  impaired  vision  for  coarse  or  intermediate  detail  (Regan, 
Silver  and  Murray,  1977;  Ginsburg,  1978;  Zimmern,  Campbell  and  Wilkinson, 

1979).  Figure  12  illustrates  the  sinewave  gratings  used  to  test  vision  at 
different  spatial  frequencies,  and  defines  "contrast"  and  "spatial  frequency".* 

Multiple  sclerosis  patients  illustrate,  in  exaggerated  form,  visual 
differences  that  also  exist  between  control  subjects.  In  studies  of  40 
control  subjects  we  found  that  those  with  20/20  vision  showed  considerable 
intersubject  variability  at  a  lower  spatial  frequency  of  5  cycles/degree 
(Regan,  Silver  and  Murray,  1977;  Regan,  Whitlock,  Murray  and  Beverley,  1980), 
so  that  a  control  subject's  ability  to  see  a  low  contrast  object  of  moderate 
size  would  not  be  accurately  predicted  by  his  Snellen  acuity,  a  point  recently 
emphasized  and  put  into  a  military  context  by  Ginsburg  (1980). 


*  Figure  12  also  illustrates  a  common  procedure  for  testing  vision  at  a  given 
spatial  frequency.  This  is  to  adjust  grating  contrast  until  the  subject  can 
just  tell  the  difference  between  the  grating  and  a  blank  field  of  similar 
brightness.  This  contrast  is  his  "contrast  threshold"  for  that  spatial 
frequency.  Then  contrast  threshold  is  measured  for  several  other  spatial 
frequencies. 


2.7  CONTRAST  THRESHOLDS 


The  smallest  contrast  for  which  the  subject  can  distinguish  between  a 
grating  and  a  blank  screen  has  been  much  studied.  However,  this  is  not  the 
only  type  of  contrast  threshold.  For  example,  there  is  a  minimum  contrast  for 
which  a  changing-size  channel  can  operate,  and  this  contrast  threshold  may  be 
somewhat  higher  than  the  contrast  required  to  see  the  target  (Petersik, 
Beverley  and  Regan,  1981).  In  other  words,  the  fact  that  a  subject  can  just 
see  a  target  does  not  necessarily  mean  that  he  can  use  all  his  various  visual 
channels  (e.g.,  changing-size,  motion,  motion  in  depth  channels):  there  may 
be  a  gap  between  the  contrast  threshold  to  just  detect  a  target  and  the 
contrast  threshold  to  activate  a  particular  channel.  For  this  reason  we 
included  measurements  of  contrast  thresholds  for  changing-size  and  motion 
channels  in  the  study  on  pilots'  flying  performance  described  below. 


3  .  VISUAL  TESTS  AND  FLYING  PERFORMANCE 


3.1  METHODS 


Psychophysical  tests:  Procedure  for  measuring  psychophysical  thresholds 

We  used  a  staircase  procedure  {Wetherill  and  Levitt,  1965)  to  make  all 
our  threshold  measurements.  Four  staircases  were  interleaved  to  prevent  the 
subject  from  deducing  the  staircase  strategy.  Consequently,  each  run  provided 
four  estimates  of  threshold,  one  for  each  of  the  four  interleaved  staircases. 
Some  estimate  of  reliability  could  be  obtained  by  comparing  these  two 
estimates.  Stimulus  delivery  was  controlled  by  a  computer  (Commodore  PET 
20011  that  also  received  the  subject's  response  ("stimulus  present'V'stimulus 
not  present"),  then  computed  and  printed  thresholds  and  standard  deviation, 
according  to  the  method  described  by  Wetherill  and  Levitt  (1965).  A  full 
account  of  methods  is  given  elsewhere  (Kruk,  Regan,  Beverley  and  Longridge, 
19811. 

Changing  size  and  motion  thresholds.  Two  1°  x  1°  solid  bright 
squares  of  luminance  6  cd/m?  were  presented  on  a  Tektronix  model  608  CRT. 

In  one  psychophysical  test  the  subject  fixated  the  centre  of  the  square  while 
the  square's  size  oscillated  at  a  frequency  of  2  Hz  (opposite  edges  oscillated 
sinusoidally  in  antiphasel.  In  a  second  test  the  square's  opposite  edges 
oscillated  sinusoidally  inphase  so  that  the  square  as  a  whole  oscillated  in 
position  along  a  diagonal. 

Contrast  thresholds  for  motion  perception.  This  test  resembled  the 
motion  threshold  tests  described  above  except  that  the  amplitude  of 
oscillation  was  held  constant  at  6  min  arc  peak-to-peak  and  the  contrast  of 
the  square  was  varied  in  successive  presentations  according  to  the  staircase 
procedure.  The  subject  was  Instructed  that  his  task  was  not  to  detect  the 
presence  of  the  square.  Instead  his  task  was  to  detect  the  square's 
oscillation. 


Is 
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Contrast  threshold  for  sinewave  grating  detection.  All  gratings 
presented  were  of  mean  luminance  6  cd/m^  ana  spatial  frequency  5  c/deg  and 
occupied  an  area  4°  x  5°.  The  subject  was  instructed  to  press  buttons 
according  to  whether  he  detected  or  did  not  detect  the  grating.  Gratings  were 
presented  according  to  the  staircase  procedure. 

Tracking  tests.  The  device  used  to  carry  out  this  test  has  been 
described  previously  (Regan  and  Beverley,  1980).  In  brief,  a  0.5°  x  0.5° 
bright  solid  square  was  presented  on  a  Tektronix  model  608  CRT.  The  square's 
luminance  was  6  cd/W?  and  it  was  superimposed  on  a  background  of  luminance 
24  cd/m^  that  subtended  10°  x  10°.  Thus,  the  contrast  of  the  square  was 
25%.  Contrast  was  checked  directly  by  placing  a  strip  of  0.6  log  unit  neutral 
density  filter  over  the  square. 

In  the  antiphase  tracking  test  the  square  started  to  expand  at  a  constant 
rate  (13  min  arc  per  sec  per  edge),  and  after  a  pseudo-random  interval  ^  it 
started  to  contract  at  the  same  rate,  and  after  a  second  pseudo-random 
interval  t  it  started  to  expand  again  at  the  same  rate,  and  so  on.  The  mean 
value  of  T, was  3.0  sec. 

The  perturbed  antiphase  tracking  task  was  similar  to  the  antiphase 
tracking  task  described  above  except  that  the  position  of  the  test  square  was 
continuously  moved  about  the  screen  by  a  noise  waveform  of  bandwidth  d.c.-0.7 
Hz  and  RMS  amplitude  22.5  min  arc.  Thus,  this  positional  perturbation  was 
quite  severe,  and  the  maximum  sideways  excursion  during  any  test  run  could  be 
several  times  the  width  of  the  square. 

About  20  min  were  required  to  complete  all  the  tracking  tests. 

In  Table  1  the  following  abbreviations  have  been  used  to  designate  the  13 
psychophysical  measures. 

TI  1  -  Inphase  tracking,  errors  0  -  0.25  Hz 

TI  2  -  Inphase  tracking,  errors  above  0.25  Hz 

TA  1  -  Antiphase  tracking,  errors  0-0.25  Hz 

TA  2  -  Antiphase  tracking,  errors  above  0.25  Hz 

TP  1  -  Perturbed  antiphase  tracking,  errors  0  -  0.25  Hz 

TP  2  -  Perturbed  antiphase  tracking,  errors  above  0.25  Hz 

SI  -  Inphase  oscillation  threshold 

SA  -  Antiphase  oscillation  threshold 

G  -  5  c/deg  grating  contrast  detection  threshold 


CPI  -  Inphase  oscillation  contrast  threshold  for  peripherally-viewed  square 
CPA  -  Antiphase  oscillation  contrast  threshold  for  peripherally-viewed  square 
CDI  -  Inphase  oscillation  contrast  threshold  for  centrally-viewed  square 
CDA  -  Antiphase  oscillation  contrast  threshold  for  centrally-viewed  square 
3.2  FLYING  TESTS 
ASPT  simulator  at  Williams  AFB 


Pilots  used  the  A-10  cockpit  to  make  12  restricted  visibility  landings  in 
the  ASPT  simulator.  Visibility  was  nominally  6000  ft  (1829  m),  3000  ft  (915 
m)  or  1500  ft  (460  m),  but  pilots  generally  agreed  that  these  nominal  figures 
corresponded  to  considerably  lower  visibilities  in  real  flying  conditions. 

The  6000-ft  visibility  condition  was  included  as  a  relatively  easy  task  so 
that  the  experiment  as  a  whole  would  not  seem  unreasonably  difficult.  In  any 
event,  there  were  very  few  crashes  in  either  the  6000-ft  or  3000-ft  visibility 
condition,  so  the  only  data  we  report  here  are  for  the  1500-ft  condition. 

Pilots  were  instructed  to  land  on  each  approach.  The  ground  could  not  be 
seen  at  the  start  of  each  approach.  Pilots  were  instructed  that  their  initial 
air  speed  was  120  kts  and  glide  slope  2.5°,  but  that  their  trajectory  was 
incorrect  in  both  centerline  alignment  (by  +200  ft,  i.e.,  +61  m),  and  range 
(by  +1000  ft,  i.e.,  +305  m).  About  20  min  practice  was  allowed  before  the 
test,  comprising  10  min  free  flying  then  four  visual  approaches  to  landing  and 
two  landing  approaches  in  each  of  three  visibility  conditions. 

The  simulator  indicated  a  crash  in  any  condition  that  would  have  resulted 
in  a  crash  with  an  A-10  aircraft. 

We  defined  the  first  visual  flight  correction  as  the  first  correction 
towards  the  runway  which  exceeded  the  mean  of  the  previous  maximum  angles  of 
bank  by  50%  or  more.  Typically,  the  ratio  was  considerably  greater.  For 
example,  a  pilot  who  used  5°-10°  angles  of  bank  during  instrument  approach 
commonly  used  up  to  30°-40°  of  bank  when  he  first  saw  the  runway. 

Flying  grades 

During  the  course  of  their  regular  training.  Group  2  students  had  been 
assessed  into  four  grades  in  the  following  aspects  of  flying  performance  in 
T37  and  T38  jet  trainers:  General  flying  performance  (FLY),  formation 
performance  (FORM),  instrument  flying  performance  (IF),  and  landing 
performance  (LAND).  The  grades  were  as  follows:  U  (unsatisfactory),  F 
(fair),  G  (good),  and  E  (excellent).  We  converted  these  letter  grades  to  a 
four  point  numerical  scale  (U  -  1,  F  =  2,  G  =  3,  E  =  4). 


322 


3.3  SUBJECT  GROUPS 


There  were  12  subjects  in  each  group.  Group  1  comprised  Air  Force 
primary  student  jet  pilots  training  on  the  T37  aircraft.  The  age  range  was 
22-28  years,  mean  23.6  years.  Group  2  comprised  Air  Force  graduating  student 
jet  pilots  training  on  the  T38  aircraft.  Their  age  range  was  23-29  years, 
mean  24.8  years.  Group  3  comprised  experienced  pilot  instructors.  Their 
range  was  26-37  years,  mean  30.3  years,  and  their  flying  hours  ranged  from 
1340-4940,  with  a  mean  of  2430.  Group  4  was  made  up  of  experienced  weapons 
officers  who  had  a  great  deal  of  nonpilot  flying  experience.  Some  members  (5) 
of  this  group  had  private  pilot  flying  experience.  Their  ages  ranged  from 
27-39  years  with  a  mean  of  32.9  years,  and  their  flying  (pilot)  hours  ranged 
from  0  to  180  with  a  mean  of  36.  Group  5  comprised  nonflying  control  subjects 
taken  from  a  University  population.  All  had  corrected  vision  of  20/20  or 
better  and  none  had  a  history  of  ophthalmological  disorder.  Their  ages  ranged 
from  19-34  years  with  a  mean  of  23.4  years. 

3.4  VISUAL  TESTS  AND  FLYING  PERFORMANCE:  RESULTS 

Correlations  between  the  13  psychophysical  measures  and  landing  performance  on 
the  ASPT  simulator 


For  Group  2  (i.e. ,  advanced  students),  good  predictions  of  low-visibility 
landing  performance,  as  assessed  by  the  number  of  crashes,  could  be  made  on 
the  basis  of  the  antiphase  tracking  task  (Table  1).  By  way  of  illustration, 
Figure  13  plots  the  relationship  between  antiphase  tracking  performance 
(ordinate)  and  the  number  of  crashes  (abscissa).  There  was  a  clear 
correlation  (r  =  0.68,  p  =  0.008). 

The  distance  from  the  runway  threshold  at  which  the  pilot  made  the  first 
flight  correction  correlated  most  strongly  with  the  results  of  the  inphase 
tracking  test.  In  particular,  the  better  the  pilot  performed  on  these  tests, 
the  further  he  was  from  the  runway  when  he  made  his  first  flight  correction. 

By  way  of  illustration.  Figure  14  plots  the  relationship  between  the  inphase 
tracking  performance  (ordinates)  and  the  distance  from  the  runway  threshold  at 
which  Group  2  pilots  made  their  first  flight  correction.  The  relationship  can 
be  seen  to  be  remarkably  close. 

In  addition  we  found  that  contrast  thresholds  for  motion  perception  and 
changing-size  perception  correlated  with  landing  performance  (CPI  and  CPA, 
Table  1).  Somewhat  surprisingly,  grating  contrast  threshold  did  not  correlate 
significantly  with  the  number  of  crashes  nor  even  with  the  distance  from  the 
runway  at  which  the  first  flight  correction  was  made. 

For  Group  3  (pilot  instructors)  correlations  were  generally  at  much  lower 
levels  of  significance  than  for  Group  2  so  that  no  firm  conclusions  could  be 
drawn.  These  low  significance  levels  might  well  have  been  due  to  intersubject 


differences  in  several  test  and  performance  measures  being  considerably 
smaller  in  Group  3  than  in  Group  2. 


Correlations  between  the  13  psychophysical  measures  and  flying  grades  on  T37 
and  T38  jet  aircraft 

Flying  grades  were  available  for  Group  2  (advanced  students)  only.  There 
were  significant  correlations  between  flying  grades  and  several  of  the 
threshold  contrasts  for  motion  perception  ctnd  these  are  reported  elsewhere 
(Kruk,  Regan,  Beverley  and  Longridge,  1981).  However,  they  should  be  treated 
with  caution  as  flying  performance  was  graded  by  subjective  evaluation  for 
purposes  other  than  this  study. 

Which  psychophysical  tests  distinguished  between  the  various  groups  of 
subjects? 


The  tracking  measures  were  far  more  successful  than  any  of  the  threshold 
measures  in  distinguishing  between  the  various  groups  of  subjects. 

The  inphase  tracking  task  (TI  1,  d.c.  -  0.25  Hz  errors)  distinguished 
between  the  different  groups  of  flying  personnel,  in  particular  groups  2  and 
3;  2  and  4;  and  (less  clearly)  1  and  2.  This  task  did  not  distinguish  between 
the  nonflying  group  and  the  flying  groups,  largely  because  the  individual 
differences  were  very  large  in  the  nonflying  group. 

On  the  other  hand,  errors  in  the  frequency  range  above  0.25  Hz 
distinguished  between  the  nonflying  control  group  (Group  5)  and  all  groups  of 
flying  personnel  at  very  high  levels  of  significance. 

As  a  whole,  threshold  measures  were  not  at  all  effective  in 
distinguishing  between  different  groups  of  subjects.  The  sole  exception  was 
that  the  grating  contrast  threshold  test  distinguished  (pi  =  0.02)  between 
Groups  3  (pilot  instructors)  and  5  (nonflying  control  subjects). 

3.5  VISUAL  TESTS  AND  FLYING  PERFORMANCE:  PRELIMINARY  CONCLUSIONS 


Prediction  of  simulator  landing  performance  on  the  basis  of  psychophysical 
tests 


Of  our  13  psychophysical  measures,  tracking  performance  related  most 
closely  to  low-visibility  landing  performance  on  the  simulator.  In 
particular,  landing  performance,  as  assessed  by  the  number  of  crashes, 
correlated  most  strongly  with  the  student  pilot's  ability  to  track  a  target 
that  appeared  to  move  in  depth.  In  addition,  the  greater  the  student  pilot's 
ability  to  track  a  target  that  moved  from  side-to-side,  the  greater  the 


distance  from  the  runway  threshold  that  he  made  his  first  flight  correction, 
and  presumably  the  greater  the  available  time  to  adjust  his  approach.* 


Contrast  threshold  for  a  5  cycle/deg  grating  relates  to  a  subject's 
ability  to  see  a  low-contrast  object  of  moderate  size.  One  might  therefore 
expect  that,  in  a  low-visibility  landing,  a  pilot  with  a  low  grating  contrast 
threshold  would  see  ground  features  earlier  than  a  pilot  with  a  high  grating 
contrast  threshold,  and  therefore  would  make  the  first  flight  adjustment 
somewhat  earlier.  Surprisingly,  this  expectation  was  not  fulfilled  by  our 
data.  Furthermore,  grating  contrast  threshold  correlated  only  weakly  with  the 
number  of  crashes  and  with  flying  grades. 

Although  our  findings  are  preliminary  and  await  verification,  perhaps  it 
is  not  premature  to  speculate  that  a  test  involving  superthreshold  vision  and 
eye-hand  coordination  (such  as  a  tracking  task)  might  be  intrinsically  a  more 
effective  predictor  of  landing  performance  than  a  threshold  detection  test. 


*  We  should  make  the  reservation  that  our  protocol  may  have  compelled  pilots 
to  fly  in  a  way  that  ran  to  some  extent  counter  to  their  training  and 
experience  of  real  flying  situations.  For  example,  pilots  were  instructed  to 
land  on  every  approach,  and  were  not  allowed  the  option  to  make  one  or  more 
passes  before  landing.  Our  aim  here  was  to  bring  out  purely  visual 
limitations  to  landing  performance.  However,  this  may  have  put  very 
experienced  pilots  at  a  disadvantage,  as  suggested  by  the  negative 
correlations  between  experience  and  number  of  crashes  in  the  instructor  pilot 
group  (Group  3).  We  should  also  note  that  artificialities  of  the  simulator 
environment  may  have  penalized  experienced  pilots. 
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CAPTIONS  TO  FIGURES 


Figure  I 

The  ratio  between  the  velocities  of  the  left  (Vl)  and  right  (Vr) 
retinal  images  provides  an  unambiguous  cue  to  the  direction  of  motion  in 
depth.  Positive  values  of  V[_/Vr  mean  that  the  retinal  images  move  in  the 
same  direction,  while  negative  values  indicate  opposed  motion. 

Figure  2 

A  -  Visual  discrimination  between  different  direction  of  motion  in 
depth.  Ordinates  plot  "the  smallest  difference  in  directions  that  can  be 
detected  for  different  trajectories  (abscissae).  The  best  discrimination 
(about  0.2°)  is  for  an  object  moving  direcfy  towards  the  nose. 

8  -  Sensitivity  curves  for  stereoscopic  motion  channels  in  the  human 
visual  system.  These  channels  are  selectively  tuned  to  the  ration  Vl/Vr 
(see  Pig.  1).  Values  of  Vj_/Vr  are  plotted  as  abscissae,  with 
corresponding  directions  in  space  indicated.  Note  that  the  narrow  cone  of 
directions  passing  between  the  eyes  is  exaggerated  in  this  plot,  so  that  the 
two  centre  channels  are  very  sharply  selective  to  the  direction  of  motion  in 
depth. 

Figure  3 

Channels  for  motion  in  depth  are  much  more  sluggish  than  channels  for 
sideways  motion.  A  -  Visual  sensitivity  to  sideways  oscillations  of  a  target 
plotted  versus  oscillation  frequency.  B  -  Visual  sensitivity  to  oscillations 
in  depth  plotted  versus  oscillation  frequency. 

Figure  4 

Spikes  recorded  from  single  neurons  in  cat  visual  cortex  (plotted 
radially)  as  a  function  of  the  direction  of  motion  in  depth  (plotted  as 
azimuthal  angle).  A  -  This  unit  maintains  its  very  selective  tuning  to  the 
direction  of  motion  in  depth  over  a  4:1  speed  range.  Note  that  firing  is 
almost  restricted  to  a  range  of  directions  little  wider  than  1°.  This 
directional  tuning  is  achieved  by  interocular  inhibition  as  shown  by  the 
arrows.  B  -  This  unit  fired  appreciably  only  when  the  target  moved  closely 
parallel  to  the  frontoparallel  plane  in  a  left-right  direction  and  when  vision 
was  binocular.  Closed  circles  show  firing  when  the  two  eyes  were  stimulated 
separately  and  open  circles  show  firing  when  binocular  vision  was  used.  The 
dotted  area  indicates  the  very  strong  interocular  facilitation  observed  for 
binocular  vision. 

Figure  5 

Experimental  evidence  for  the  existence  of  changing-size  channels. 

This  figure  shows  selective  threshold  elevations  for  changing  size.  The  two 
test  stimuli  are  shown  at  the  top.  Ordinates  plot  loss  of  visual  sensitivity 
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to  oscillation  caused  by  inspecting  a  strong  2-Hz  oscillation  of  size. 
Changing-size  sensitivity  fell  considerably,  but  sensitivity  to  oscillating 
position  was  little  affected.  Note  that  the  only  difference  between  the  two 
test  stimuli  was  in  the  relationship  between  the  motion  of  opposite  edges  of 
the  square. 

Figure  6 

Experimental  evidence  that  the  changing-size  channel  extracts  the 
vel oc Uy  component  along  tHe"'line  of  sight.  "A  -  Experimental  rationale .  All 
tra’jeciLories  had  the  same  inphase  oscillation  component  of  6  min  arc  pk-pk, 
but  different  amplitudes  of  antiphase  component  were  added.  B  -  Threshold 
elevations  produced  by  adapting  to  the  11  trajectories  in  A.  Antiphase  test 
oscillations  are  dotted,  while  the  continuous  line  plots  inphase  test 
oscillations.  C  -  An  8-Hz  "jitter"  oscillation  was  added  to  the  2-Hz  adapting 
oscillation  of  B. 

Figure  7 

Model  of  changing-size  channel. 

Figure  8 

The  changing-size  channel  computes  the  component  of  an  object's  velocity 
along  the  line  of  sight  (Vlqs)  even  when  both  object  and  observer  are  moving 
along  arbitrary  tra jectories.  In  the  figure  V^gj  =  Vy  cosine  a  +  Vg 
cosine  B. 

Figure  9 

When  a  real-world  nonrotating  object  moves  closer,  the  retinal  images 
grow  larger  and  move  apart. 

Figure  10 

Cancellation  of  motion-in-depth  sensation  by  opposing  changing-size  and 
changing-disparity  stimuli:  effect  of  velocity.  The  subject  viewed  a  square, 
each  of  whose  edges  moved  outwards  at  constant  speed  for  either  0.25,  1.0  or 
3.3  sec  (as  marked  on  figure).  .The  square  then  disappeared  for  0.25  sec  and 
the  cycle  repeated.  Abscissae  9$,  the  speed  of  outward  motion  of  outward 
motion  of  each  edge  in  min  arc  sec  “1  (where  2  6$  is  the  rate  of  increase 
of  a  square] s  side  length).  Ordinates  plot  the  rate  of  change  of  binocular 
disparity  IQg  min  arc  sec  "')  required  to  cancel  the  sensation  of  motion 
in  depth  produced  by  the  fixed  rate  of  change  of  size  0$. 
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Figure  11 


Threshold  versus  oscillation  frequency  curves  for  changing-size 
stimulation  producing  motion-in-depth  sensation  (continuous  line), 
changing-size  stimulation  producing  changing-size  sensation  (fine  dotted  line) 
and  changing-disparity  stimulation  producing  motion-in-depth  sensation  (bold 
broken  line). 

Figure  1? 


A  and  B  illustrate  the  sinewave  gratings  used  to  test  visual 
contrast  sensitivity.  A  is  a  high  contrast  grating  (about  50%)  and  B  is  a 
fairly  low  contrast  grating  (about  10%).  C  shows  the  variation  of  light 
intensity  across  the  screen  for  a  grating  of  almost  100%  contrast  (continuous 
line)  and  for  one  of  about  20%  contrast  (broken  line).  Percentage  contrast  is 
defined  as  100  a/b.  Contrast  threshold  is  the  percent  contrast  for  which 
the  subject  can  just  see  the  grating. 

Figure  13 

Relation  between  tracking  performance  and  the  number  of  crashes. 

Ordinates  plot  error  in  tracking  a  test  square  whose  size  continuously  and 
unpredictably  changed  as  though  moving  in  depth.  Errors  in  the  frequency 
range  below  0.25  Hz  only  are  shown.  Abscissae  plot  the  number  of  crashes  made 
in  four  low-visibility  landings  on  the  ASPT  simulator  (A-10  cockpit).  Data 
for  12  graduating  student  pilots  are  shown. 

Figure  14 

Relation  between  tracking  performance  and  the  distance  from  the  runway 
threshold  at  which  pilots  made  the  first  flight  correction.  Ordinates  plot 
errors  in  tracking  a  test  square  that  moved  unpredictably  in  the 
frontoparallel  plane.  Errors  in  the  frequency  range  below  0.25  Hz  only  are 
shown.  Abscissae  plot  the  distance  from  runway  threshold  at  which  the  first 
flight  correction  was  made.  Negative  distances  mean  that  the  pilot  had  passed 
over  the  threshold  before  his  first  correction.  Results  are  the  mean  for  four 
low-visibility  landings. 
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Table  1 


SIMULATOR  PERFORMANCE  AND  VISUAL  TESTS 


Subject 

Groups  Correlations  Between  r  p 


2+3 

crashes 

& 

TA  1 

0.43 

0.02 

2+3 

crashes 

& 

TP  1 

0.49 

0.008 

2+3 

crashes 

& 

G 

-.24 

.13 

2+3 

correction 

& 

TI  1 

0.43 

0.02 

2+3 

correction 

& 

G 

-.02 

.46 

3 

crashes 

& 

hours 

-.35 

.13 

3 

crashes 

& 

age 

-.43 

.08 

3 

crashes 

& 

IF 

-.16 

.30 

1 

TA2 

& 

hours 

-.56 

.03 

2 

correction 

& 

TI  1 

0.7 

0.006 

2 

crashes 

& 

TA  1 

0.68 

0.008 

2 

crashes 

& 

TP  1 

0.53 

0.04 

2 

crashes 

& 

SA 

0.68 

0.008 

2 

crashes 

& 

CPI 

0.61 

0.02 

2 

crashes 

& 

CPA 

0.51 

0.05 

FLYING  PERFORMANCE  AND  VISUAL  TESTS 

2 

TA  1 

& 

FIF 

0.56 

0.03 

2 

SA 

& 

FLY 

0.50 

0.05 

2 

CPI 

& 

FLY 

0.56 

0.03 

2 

CPI 

& 

FORM 

0.58 

0.02 

2 

CPA 

& 

FLY 

0.55 

0.03 

2 

CPA 

& 

FORM 

0.59 

0.02 

2 

CDI 

& 

FLY 

0.63 

0.01 

2 

CDI 

& 

LAND 

0.57 

0.03 

FLYING  GRADE  AND 

FLYING  GRADE 

2 

FLY 

& 

FORM 

0.64 

0.01 

2  FLY  &  LAND  0.61  0.02 

2  FORM  &  LAND  0.82  0.0005 


Correlation  between  visual  test  results  and  simulator  performance. 

All  p  values  not  shown  were  less  than  0.05.  The  abbreviations  are  explained 
on  page  19.  "Hours"  is  total  number  of  flying  hours,  and  IF  is  total  number 
of  instrument  flying  hours. 
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Hughes  Aircraft  Company  and  the  National  Institutes  of  Health,  followed  with 
academic  involvement  in  robotics  at  California  Institute  of  Technology,  and 
for  the  past  six  years,  research  into  human  vision  at  the  M.I.T.  Artificial 
Intelligence  Laboratory.  His  interests  are  primarily  in  surface  perception 
and  the  detection  and  encoding  of  texture  and  contour  Information. 


DEBATERS,  CON 


Dr.  Herschel  W.  Leibowitz 
Evan  Pugh  Professor  of  Psychology 
The  Pennsylvania  State  University 


Dr.  Leibowitz  received  his  Ph.D.  degree  in  Psychology  under  the  guidance 
of  Professor  C.  H.  Graham.  He  has  published  in  the  field  of  vision  and 
holds  numerous  positions  of  high  scientific  stature.  He  is  presently  Evan 
Pugh  Professor  of  Psychology  at  Pennsylvania  State  University. 

Dr.  Josh  Y.  Zeevi 
Man-Vehicle  Laboratories 
Massachusetts  Institute  of  Technology 
Cambridge,  MA 


Yehoshua  Y.  Zeevi  obtained  is  Ph.D.,  M.S.,  and  B.S.E.E.  from  the 
University  of  California  at  Berkeley,  the  University  of  Rochester  and  the 
Technion-Israel  Institute  of  Technology  respectively.  His  professional 
experience  has  been  in  the  academia,  mostly  with  the  Technion,  Harvard 
University,  Lawrence  Berkeley  Lab.,  and  M.I.T.  He  has  worked  extensively 
with  display  systems  studying  signal  detection,  eye  movement 
characterisitics  and  visual  information  processing.  He  is  presently  a 
visiting  Professor  at  the  Man-Vehicle  Lab.,  Department  of  Aeronautics  and 
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FLIGHT  SIMULATOR  VISUAL  AND 
INSTRUCTIONAL  FEATURES 
FOR  TERRAIN  FLIGHT  SIMULATION 


George  H.  Buckland,  Bernell  3.  Edwards,  and  Clarence  W.  Stephens 
Air  Force  Human  Resources  Laboratory 
Operations  Training  Division 
Williams  AFB,  Arizona 


Lack  of  adequate  visual  scene  detail  is  a  serious  limitation  in  the 
use  of  computer  generated  imagery  for  training  low  altitude  terrain 
flight.  Terrain  flight  profiles  involve  both  low  level  contour  flight 
which  closely  follows  the  profile  of  the  ground  and  nap-of-the-earth  (NOE) 
flight  which  includes  maneuvering  between  and  around  vertical 
obstructions.  Both  ground  textural  pattern  cues  and  vertical  object  cues 
appear  to  be  important  for  pilots  to  judge  aircraft  height  above  the 
ground.  AFHRL/OT  has  conducted  research  studies  on  both  of  these  types  of 
visual  cues  and  is  currently  developing  an  optimal  ASPT  (Advanced 
Simulator  for  Pilot  Training)  visual  environment  for  instructing  terrain 
flight  maneuvers. 

The  initial  research  study  of  visual  information  cues  for  pilot  depth 
perception  investigated  T-37  pilot  landing  performance  in  response  to  four 
different  checkerboard-like  texture  patterns  superimposed  upon  the 
simulated  runway  touchdown  zone  area.  The  check  sizes  used  were  4,  8,  16, 
or  25  feet  on  a  side  (Figure  1).  Two  additional  daytime  runways  also  were 
used  consisting  of  a  simulated  Air  Force  runway  with  standard  markings  and 
a  completely  bare  runway  with  no  markings  except  for  the  dashed 
centerline.  A  night  runway  scene  was  also  used  bringing  the  total  number 
of  runways  tested  to  seven.  In  this  study  the  average  vertical  velocities 
at  touchdown  ranged  from  195  feet/minute  for  the  night  scene  to  147 
feet/minute  for  the  four-foot  texture  pattern.  Although  the  simulated 
aircraft  vertical  velocities  at  touchdown  were  much  higher  than  those 
recorded  during  actual  aircraft  landing  (32  ft/minute),  the  textured 
runway  patterns  did  significantly  reduce  simulated  aircraft  vertical 
velocities  at  touchdown.  Interestingly,  this  reduction  in  touchdown 
vertical  velocity  occurred  in  a  near  linear  progression  as  a  function  of 
Increasing  texture  detail  (Table  I). 
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Table  1 


T-37  AIRCRAFT  VERTICAL  VELOCITY  AT  TOUCHDOWN 


Simulated 

Without 

With 

(Ft/Min) 

Visual  Scene 

Overrun 

Overrun 

Average 

Night 

201 

190 

195 

Bare  Bones 

195 

171 

183 

Willy 

172 

179 

175 

25  Ft  Texture 

168 

168 

168 

16  Ft  Texture 

158 

173 

165 

8  Ft  Texture 

151 

170 

161 

4  Ft  Texture 

136 

158 

147 

Edwards  Flight  Test  Center  Cine-Theodolite  32  Ft/Min 

Tracking  of  T-37  Landings 

In  a  subsequent  study  AFHRL/OT  investigated  the  effects  of  three  types 
of  visual  cues,  texture  patterns  (checkerboards,  220,  440  or  880  feet  on  a 
side),  vertical  objects  (present  or  absent)  and  aircraft  shadow  (present 
or  absent),  upon  pilot  performance  during  NOE  flight  In  a  simulated  A-10 
aircraft  (Figure  2).  In  this  study  pilots  were  instructed  to  fly  at  50 
feet  above  the  ground  on  a  ten  mile  course  which  consisted  of  eight 
valleys  which  were  separated  by  low  rolling  hills  that  were  either  100  or 
300  feet  high.  The  pilots  were  scored  on  their  ability  to  maintain 
aircraft  altitude  at  50  feet  plus  or  minus  30  feet  in  the  flat  valleys. 
Actual  aircraft  values  were  also  collected  at  the  crest  of  each  hill.  The 
pilots  flew  the  course  at  300  knots  indicated  airspeed  plus  or  minus  15 
knots. 

In  general  the  pilots  reported  that  all  three  types  of  visual  cues 
were  useful,  however,  the  vertical  object  cues  and  texture  patterns  were 
of  greater  help  than  the  aircraft  shadow.  Some  pilots  reported  that  the 
aircraft  shadow  was  particularly  useful  in  signalling  impending  contact 
with  the  ground.  The  vertical  object  cues,  especially  the  tree  shaped 
cones  of  known  height,  were  subjectively  very  useful  in  gauging  height 
above  ground.  The  texture  patterns  were  also  reported  as  desirable,  but 
to  a  lesser  extent  than  vertical  object  cues.  Pilots  reported  a  definite 
preference  for  the  smallest  texture  pattern  (220  foot  square  size)  over 
the  two  larger  size  patterns.  They  especially  disliked  flying  over  the 
largest  texture  pattern  without  vertical  object  cues.  Pilots  would  have 
also  preferred  more  irregular  "natural"  patterns  rather  than  the  highly 
regular  checkerboard  features.  Both  the  texture  patterns  and  the  vertical 
object  cues  produced  statistically  significant  differences  in  pilot 
performance.  However,  only  the  texture  pattern  cues  produced  a 
significant  effect  on  the  time  within  tolerance  scoring  for  altitude  in 
the  valleys,  and  the  average  minimum  altitude  values  in  the  valleys 
(Tables  2  &  3).  The  vertical  object  cues  did  have  a  significant  effect  on 
the  average  aircraft  altitude  at  the  top  of  the  hills  (Table  4).  Both  the 
vertical  object  cues  and  the  texture  patterns  significantly  influenced  the 
average  minimum  altitude  values  that  occurred  over  each  hill  (Table  5). 
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Figure  2  (continued) 
NOE  I  Visual  Environment 
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Table  2 


%  Time  Within  Tolerance  Scoring 
for 

Aircraft  Altitude  in  Valleys  (50  +  30  Ft.) 


*  Texture  Vertical  Aircraft 

Patterns  (Ft)  Objects  Shadow 


220  440  880 _ With  Without _ With  Without 

77.4  72.6  64.7  72.2  70.9  71.1  72.0 


*  P  .001 


Table  3 


Avg.  Min.  Altitude  Values  in  Valleys 


*  Texture 

Vertical 

Aircraft 

Patterns  (Ft) 

Objects 

Shadow 

220  440  880 

With  Without 

With 

Without 

43.0  45.5  47.0 

44.8  45.5 

45.7 

44.6 

*  P  .003 
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Table  4 


Avg.  Aircraft  Altitude  (Ft)  at  Hill  Tops 


*  Texture 
Patterns 

(Ft) 

Vertical 
*  Objects 

Aircraft 

Shadow 

220 

440 

880 

With 

Without 

With 

Without 

67.8 

67.6 

69.2 

63.6 

72.7 

69.3 

67.1 

*  P  .001 


Table  5 

Avg.  Aircraft  Min.  Altitude  (Ft)  Over  Hills 


Texture 

Vertical 

Aircraft 

*  Patterns  (Ft) 

*  Objects 

Shadow 

220  440  880 

With  Without 

With 

Without 

40.6  45.9  47.9 

41.4  48.2 

45.3 

44.3 

*  P  .001 
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The  presence  or  absence  of  the  aircraft  shadow  did  not  produce  any 
significant  effects.  The  amount  of  time  (cumulative  total)  in  a  "crashed" 
condition  (in  contact  with  ground)  was  low,  with  some  pilots  crashing  into 
simulated  terrain  more  frequently  than  others.  No  significant  differences 
were  found  for  this  performance  parameter  due  to  visual  cue  variables. 
Overall,  in  this  study,  the  terrain  textural  cues  appeared  to  have  a 
stronger  effect  on  pilot  performance  than  did  the  vertical  object  cues. 

Since  the  development  of  the  original  NOE  I  environment  researchers  at 
AFHRL/OT  have  found  that  cone  (actually  tetrahedron)  shaped  "trees"  appear 
to  provide  improved  visual  cueing  when  turned  upside  down  so  that  the 
broader  base  is  more  directly  visible  to  the  pilots.  This  technique 
provides  a  very  edge-efficient  (6  edges  total)  visual  cue  which  has 
effectively  reduced  terrain  crash  frequency  during  low  level  flight  in  our 
hostile  threat  simulation  environments  (Figure  3). 

Our  most  recent  terrain  flight  environment,  named  NOE  II,  uses  both 
inverted  cones  and  random  ground  texture  patterns.  Pilots  subjectively 
report  that  this  is  the  best  ASPT  visual  environment  yet  for  terrain 
flight.  NOE  II  is  modeled  to  approach  as  nearly  as  CIG  capacity  permits, 
the  irregular  natural  features  likely  to  be  seen  in  actual  terrain  flight 
(Figure  41.  The  available  edges  are  concentrated  within  and  adjacent  to 
the  intended  flight  path  in  order  to  maximize  available  cues  to  the 
pilot.  The  flight  path  is  bordered  by  hills  which  slope  away  from  the 
path  at  realistic  rise  angles. 

The  NOE  II  environment  will  be  used  to  study  the  combination  of 
instructional  techniques  with  display  content  variations  to  produce 
optimal  training  effects  for  terrain  flight.  With  regard  to  vertical 
objects,  inverted  cones  of  several  heights  will  be  represented  with  the 
shape  of  the  cone  proportioned  to  the  expected  shape  of  a  similar  tree. 
Three  tree  heights  (25,  40,  and  55  feet)  will  be  used  with  the  25  foot 
tree  rather  short  and  squat,  the  55  footer  tall  and  thin,  and  the  40  foot 
tree  of  medium  proportion  so  that  height  can  be  easily  distinguished  using 
proportion  as  a  cue.  The  placement  and  density  of  vertical  objects  will 
also  be  investigated  with  regard  to  effective  altitude  cueing  and 
navigational  check  points  along  the  route. 

With  regard  to  instructional  variables,  in  one  study  various 
combinations  visual  and  auditory  cues  will  be  provided  to  the  pilot  to 
test  their  effects  on  the  acquisition  of  terrain  following  skills.  These 
ancillary  cues  will  be  faded  out  as  desired  student  pilot  responses  are 
established.  In  another  study,  variations  of  feedback  associated  with 
terrain  strikes  will  be  investigated  to  determine  the  content  and  timing 
of  feedback  as  a  means  of  reducing  terrain  impact  and  increasing  low  level 
skills.  "Training  down"  sequences  will  also  be  tested  in  which  pilots 
inexperienced  in  low  altitude  flight  will  be  presented  with  successively 
more  difficult  altitude  ceilings  as  they  progress  toward  NOE  flight 
proficiency.  Other  instructional  variables  are  also  under  consideration, 
including  the  use  of  slow  flight  during  early  trials,  concomitant 
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secondary  information  loading  to  vary  task  difficulty,  and  pretraining  of 
environmental  features  and  flight  techniques  in  order  to  establish 
perceptual  set/expectancies. 

Several  dependent  measures  of  pilot  performance  will  be  used  for  the 
NOE  II  studies,  including  relevant  aircraft  state  parameters  sampled 
during  selected  portions  of  flight,  time  within  tolerance  scoring  for 
altitude  and  airspeed,  and  terrain  strike  frequency,  duration,  and 
aircraft  position.  Initial  data  collections  for  NOE  II  studies  are 
planned  for  the  Spring  and  Summer  of  1981.  We  expect  to  report 
preliminary  results  by  the  Fall  of  1981. 
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Figure  4 

NOE  II  Environment 
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A  Model-Based  Procedure  for  Determining  Visual  Cue  Requirements 
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A  Model-Based  Procedure  for  Determining  Visual  Cue  Requirements 

by 

Greg  L.  Zacharias  and  William  H.  Levison 


ABSTRACT 


This  paper  outlines  a  model-based  procedure  for  determining  visual  cue 
requirements  for  flight  simulators.  Issues  addressed  by  the  procedure  include: 
(1)  determination  of  task-relevant  cues  available  from  the  visual  scene;  (2)  the 
quality  of  the  information  available  from  such  cues;  (3)  the  cue  set  that  the 
pilot  should  use  in  performing  a  specific  mission;  and  (4)  the  degree  of  fidelity 
with  which  the  various  cues  must  be  simulated.  The  optimal-control  model  (OCM) 
for  pilot/vehicle  systems  provides  the  framework  for  the  proposed  methodology. 

As  this  model  incorporates  both  the  state-estimation  and  the  control-response 
characteristics  of  the  pilot,  the  procedure  may  be  applied  to  monitoring  as  well 
as  to  continuous  control  tasks. 
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Introduction 


Recent  advances  in  digital  computing  technology  have  increased  the 
capabilities  of  aircraft  simulators  for  providing  realistic  presentations  of 
extra-cockpit  visual  scenes.  But  with  this  increase  in  capability  has  come 
an  increase  in  cost  and  complexity,  and  thus,  as  in  the  past,  designers  of 
simulators  are  still  confronted  with  difficult  choices  between  simulation 
fidelity  and  the  need  for  cost-effective  methods  of  simulation. 

The  specification  of  fidelity  requirements  for  cueing  systems  is  very 
difficult,  however,  for  several  reasons.  The  requirements  are  governed  by  the 
purpose  of  the  simulation:  training  simulators  have  different  needs  than  those 
of  research  simulators.  They  are  also  problem  dependent  (e.g.,  the  need  for 
motion  cues  for  aircraft  control  in  a  gusty  environment  will  depend  on  the 
gust  response  of  the  aircraft).  Finally,  the  specification  involves  complex 
psychophysical  as  well  as  engineering  factors.  Not  only  is  there  a  need  to 
understand  and  account  for  the  perceptual  capabilities  and  limitations  of  a 
human  pilot,  but  also  one  must  account  for  the  fact  that  the  adaptive  pilot 
may  be  able  to  compensate  for  simulator  shortcomings,  while  maintaining  system 
performance  at  the  expense  of  workload. 

These  problems  point  to  the  need  for  understanding  how  the  pilot  utilizes 
the  cues  available  to  him,  for  effective  execution  of  the  given  simulated 
flight  task.  Recognizing  this  need,  and  anticipating  the  eventual  widespread 
use  of  sophisticated  visual  scene  simulations,  we  have  chosen  to  demonstrate 
how  this  understanding  might  be  achieved  through  the  use  of  human  operator 
modelling  technology,  which  focusses  on  visual  cueing  issues.  This  paper 
thus  outlines  an  analytic  and  experimental  effort  aimed  at  developing  and 
validating  a  quantitative  model  of  the  pilot’s  use  of  extra-cockpit  visual 
cues . 


Our  analytic  effort  is  centered  on  the  Optimal  Control  Model  (OCM)  of 
the  pilot,  which  provides  the  overall  framework  for  relating  cue  integration 
to  task  performance.  Our  complementary  experimental  effort  is  chosen  to 
satisfy  two  objectives:  a)  quantifying  the  independent  parameters  of  the  OCM 
that  define  the  pilot's  ability  to  utilize  extra-cockpit  visual  cues;  and 
b)  validating  the  ability  of  the  OCM  in  predicting  pilot/vehicle  performance 
in  tasks  involving  multiple  real-world  visual  cues.  This  report  summarizes 
our  analytic  modelling  effort,  and  outlines  an  experimental  program  for  model 
development  and  validation. 

Optimal  Control  Model 

The  OCM  is  a  model  of  the  man-machine  system,  based  on  the  assumption  that 
the  well-trained,  highly-motivated  operator  will  adopt  an  optimal  response 
strategy,  subject  to  task  requirements  and  subject  to  the  operator's  Inherent 
information-processing  limitations.  Inputs  to  this  model  include  system 
dynamics,  task  requirements  (in  terms  of  a  quadratic  "cost  functional"  to  be 
minimized),  and  human-controller  limitations.  Model  outputs  include  perform¬ 
ance  metrics  such  as  means  and  variances  for  task  variables,  operator  response 
strategy,  and  operator  "noise"  characteristics. 
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The  reader  is  referred  to  the  literature  (e.g.,  Kleinman  et  al,  1971; 
Levison  et  al,  1969;  Baron  et  al,  1970)  for  a  full  description  of  the  mathe¬ 
matical  development  and  validation  of  the  model.  Here,  those  aspects  of  the 
model  that  are  most  relevant  to  display  analysis,  as  shown  in  Figure  1,  are 
discussed. 


perceptual  model  processing  mooel 


Figure  1.  Information  Processing  Model 


Display  Variables 

Let  the  "state"  of  the  controlled  element  be  represented  by  a  set  of 
state  variables  x^t),  i-l,...,n.  In  the  OCM,  it  is  assumed  that  the  operator 

is  "displayed"  outputs  y .,..., y  where  each  output  is  a  linear  combination  of 
the  state  variables:  m 

n 

y  (t)  «  l  c  (t)  x  (t)  ,  i**l . m  (1) 

j=l  2  3 


or,  in  vector-matrix  notation. 


jjr(t)  =  C(t)x(t)  .  (2) 

If  a  quantity  is  displayed  explicitly  to  the  operator  (say,  by  an  indicator 
displacement) ,  it  is  assumed  that  the  human  also  perceives  the  rate  of  change 
of  the  quantity.  Typically,  it  is  assumed  that  higher  derivative  information 
is  not  available. 

Equation  2  is  very  general  with  respect  to  representing  display  variables. 
Basic  status  information  can  be  included,  for  example,  by  letting  output 
variables  be  selected  state  variables.  Display  quickening  is  easily  modelled 
by  letting  an  output  be  a  linear  combination  of  error  and  error-rate  state 
variables  (Hess  and  Wheat,  1976).  Flight  director  signals  can  also  be 
expressed  as  linear  combinations  of  appropriately  defined  state  variables 
(Baron  and  Levison,  1973).  Note,  too,  that  the  coefficients  c^  can  be 

functions  of  time  so  that  programmed  scale  changes  or  variable  changes  may  be 
Included  in  the  problem  formulation. 

Treatment  of  these  types  of  explicit  "instrument"  displays  is  fairly 
direct,  but  additional  modelling  work  is  required  to  define  the  informational 
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base  provided  by  an  extra-cockpit  visual  scene.  We  will  discuss  this  shortly 
after  reviewing  the  additional  information-processing  components  of  the  OCM. 

Perceptual  Model 

Limitations  on  the  pilot's  ability  to  process  information  "displayed"  to 
him  are  accounted  for  in  an  "equivalent"  perceptual  model.  This  model  trans¬ 
lates  the  displayed  variables  into  delayed,  "noisy"  perceived  variables  y 
via  the  relation  ** 

Ip(t)  =  y.(t-x)  +  Vy(t-x)  (3) 

where  t  is  an  "equivalent"  perceptual  delay  and  v  is  an  "equivalent"  observa¬ 
tion  noise  vector.* 


The  various  internal  time  delays  associated  with  visual,  vestibular, 
central  processing  and  neuro-motor  pathways  are  combined  and  conveniently  rep¬ 
resented  by  this  lumped  equivalent  perceptual  time  delay  r.  Typical  values  for 
this  delay  are  0.2+. 05  sec.  (Kleinman  et  al  1971). 

The  zero-mean,  white,  Gaussian  observation  noise  vector  v  is  included  to 

~y 

account  for  the  pilot's  inherent  randomness,  due  to  random  perturbations  in 
human  response  characteristics,  errors  in  observing  displayed  variables,  and 
attention-sharing  effects  which  limit  the  pilot's  ability  to  accurately  process 
all  the  cues  simultaneously  available  to  him.  In  combination  with  the  OCM's 
motor  noise  model  (see  Kleinman  et  al, 1971).  the  observation  noise  model 
provides  a  convenient  and  accurate  means  of  modelling  pilot  remnant  and  thus 
accounting  for  random  control  actions. 

For  manual  control  situations  in  which  the  displayed  signal  is  large  enough 
so  that  visual  resolution  ("threshold")  limitations  are  negligible,  the  auto¬ 
covariance  of  each  observation  noise  component  appears  to  vary  proportionally 
with  mean-squared  signal  level.  In  this  situation,  the  autocovariance  of  the 
noise  associated  with  the  ith  display  component  may  be  represented  as 

v  i  =  (4) 

2 

where  is  the  variance  of  the  ith  display  variable,  and  is  the  "noise/ 

signal  ratio"  for  the  ith  display  variable,  which  has  units  of  normalized  power 
per  rad/sec.  Numerical  values  for  of  0.01  (i.e.,  -20  dB)  have  been  found  to 

be  typical  of  single- variable  control  situations  (Levison  et  al,  1969;  Kleinman 
et  al,  1971). 

The  perceptual  model  defined  by  (3)  and  (4)  applies  to  "ideal"  display 
conditions,  in  which  the  signal  levels  are  large  with  respect  to  both  system- 
imposed  and  pilot-associated  threhsolds.  To  account  for  threshold  effects  we 
let  the  autocovariance  for  each  observation  noise  process  be 


The  use  of  the  word  equivalent  in  this  context  is  to  emphasize  that  the 
parameters  may  be  lumped  representations  of  a  variety  of  limitations  that 
cannot  be  "identified"  separately  by  existing  measurement  techniques. 
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(5) 


where  the  subscript  i  refers  to  the  ith  display  variable.  The  quantity 
K^°i,ai^  is  describing  function  gain  associated  with  a  threshold  device 


K(a,a) 


dx 


(6) 


where  "a"  is  the  threshold  and  a  is  the  standard  deviation  of  the  "input"  to 
the  threshold  device.*  The  net  result  of  this  type  of  describing  f miction 
model  is  to  increase  the  observation  noise  covariance  as  the  display. signal 
variance  becomes  smaller  relative  to  the  threshold.  The  quantity  is  a 

"residual"  noise  term  which  is  introduced  to  account  for  performance  degrada¬ 
tion  that  arises  when  an  explicit  zero-error  reference  is  lacking  (Levison, 
1971). 


The  sources  of  these  threshold  effects  depend  on  the  particular  task  being 
modelled.  They  may  be  associated  with  the  system  display  implementation,  for 
example,  due  to  resolution  limitations  on  a  display  screen.  Or,  they  may  be 
associated  with  the  pilot's  sensory  limitations,  such  as  one  might  identify  with 
visual  acuity  thresholds. 


One  additional  factor  which  tends  to  increase  the  observation  noise 
(present  on  any  given  display  variable)  is  the  pilot’s  attention-sharing  limita¬ 
tions.  Because  the  numerical  value  associated  with  the  pilot's  noise/signal 
ratio  (P  )  has  been  found  to  be  relatively  invariant  with  respect  to  system 
dynamics  and  display  characteristics  in  single  display  situations,  we  associate 
this  parameter  with  limitations  in  the  pilot's  overall  information-processing 
capability.  This  forms  the  basis  for  a  model  for  pilot  attention-sharing  where 
the  amount  of  attention  paid  to  a  particular  display  is  reflected  in  the  noise/ 
signal  ratio  associated  with  information  obtained  from  that  display  (Levison  et 
al,  1971;  Baron  and  Levison,  1973).  Specifically,  the  effects  of  attention¬ 
sharing  are  represented  as 


P  /f . 
o  i 


where  P^  is  the  noise/signal  ratio  associated  with  the  ith  display.  When 

attention  is  shared  among  two  or  more  displays,  f^  is  the  fraction  of  attention 

allocated  to  the  ith  display,  and  P  is  the  noise/signal  ratio  associated  with 
full  attention  to  the  task.  u 

*For  non-zero-mean  signals  this  expression  must  be  modified  (see  Baron  and 
Levison,  1973). 
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Processing  Model 

The  processing  model  shown  in  Figure  1  consists  of  a  tandem  estimator/ 
predictor.  The  estimator  acts  on  the  perceived  display  vector  £^(t)  to  generate 

a  delayed  estimate  of  system  state  x(t-x),  where  x  is  the  delay  introduced  by 
the  perceptual  portion  of  the  model;  the  predictor  then  compensates  for  the 
delay  to  provide  an  up-to-date  estimate  x(t).  The  general  expressions  for 
these  model  elements  are  determined  by  system  dynamics  and  task  objective 
according  to  well-defined  mathematical  rules  that  are  derived  in  Kleinman  et 
al>  1971.  The  result  is  a  minimum  variance  estimate  of  the  system  state,  x, 
which  can  then  be  used  for  either  closed  loop  control  actions  (e.g.,  fight" 
control),  or  open-loop  decision-making  (e.g.,  failure  detection). 


Visual  Scene  Analysis  Methodology 

With  this  brief  overview  of  the  OCM,  we  can  now  consider  how  extra-cockpit 
visual  cueing  can  be  treated  within  the  model  context. 

Relevant  Visual  Cues 

In  contrast  to  the  relatively  well-defined  set  of  visual  cues  provided  by 
within-cockpit  instrumentation,  the  extra-cockpit  visual  scene  can  provide  the 
pilot  with  an  exceptionally  rich  stimulus  environment,  even  for  a  relatively 
simple  scene.  Attempting  to  describe  and  quantify  this  stimulus  environment  has 
been  the  object  of  many  studies.  For  example,  Brown  (1973)  discussed  five 
"dimensions"  of  the  visual  world:  f ield-of-view,  range  of  luminance,  color, 
spectral  resolution,  and  visual  motion.  Staples  (1970)  listed  fifteen  factors 
that  are  present  in  the  visual  scene  which  can  be  of  importance  to  the  pilot, 
and  noted  that  this  list  was  incomplete.  Gibson  (1950,1955)  concentrated  on 
geometric  and  textural  cues,  but  also  noted  the  potential  utility  of  the 
"traditional"  cues  for  depth  perception  (i.e.,  lens  accommodation,  binocular 
convergence,  and  retinal  disparity). 

The  literature  on  scene  attributes  is  extensive,  and  some  narrowing  of 
focus  is  called  for,  if  a  successful  attempt  is  to  be  made  in  the  area  of 
modelling  this  type  of  cue  processing. 

Matheny  et  al  (1971)  present  a  taxonomy  of  cues  that  is  helpful  in  struc¬ 
turing  and  limiting  the  problem.  They  define  relevant  cues  as  cues  which  are 
directly  useful  for  controlling  the  aircraft  or  for  making  decisions.  The 
non-relevant  cues  are  those  that  are  not  essential  to  the  successful  operation 
of  the  aircraft,  but  which  may  add  realism  or  face-validity  to  the  task. 

It  seems  clear  that  in  extending  the  optimal  control  model  to  account  for 
perception  of  the  visual  scene,  our  first  concern  should  be  the  relevant  cues. 
Thus,  for  example,  color  can  be  neglected,  since  adequate  foreground-background 
separation  can  be  provided  by  a  sufficiently  large  contrast  differential. 
Similarly,  range  of  luminance  might  be  neglected  if  adequate  surface  definition 
is  provided  by  the  display  hardware.  Other  "non-geometric"  visual  cue  factors 
can  be  similarly  neglected,  at  least  f ci  an  initial  analysis,  so  that  model¬ 
ing  can  concentrate  on  the  basic  geometric  characteristics  of  the  visual  scene. 
These  were  identified  by  Gibson  (1950)  in  his  analysis  of  visual  scene  proces¬ 
sing:  Table  1  lists  these  "geometric"  cues  and  the  corresponding  type  of 
information  they  provide  to  a  pilot  concerning  the  state  of  his  vehicle. 
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lable  1  Geometric  Visual  Cues 


Cue _  _ ,  Information 


Field  Orientation 

Attitude,  Attitude  Rate 

Linear  Perspective 

Position 

Motion  Parallax 

Linear  Velocity 

Apparent  Size/Range 

Position 

(Looming) 

Occultation 

Position 

Field  orientation  provides  the  pilot  with  attitude  cues:  assuming  that 
the  visual  world  is  inertially  fixed,  then  any  rotation  of  the  scene  elements, 
measured  with  respect  to  the  vehicle  (or  pilot)  frame  of  reference,  must  be 
due  to  self-rotation  of  the  vehicle  (and  pilot).  Linear  perspective  changes 
with  the  location  of  the  vehicle;  thus,  these  cues  provide  the  pilot  with 
positional  information.  Motion  parallax  is  effectively  a  rate  of  change  of 
linear  perspective,  and  thus  provides  the  pilot  with  cues  for  inferring  linear 
self-velocity.  The  apparent  size  of  an  object  (combined  with  the  pilot's 
perceptual  set  which  ensures  size  constancy)  provides  a  "looming"  or  range  cue 
which  can  be  used  to  infer  relative  distances,  and  hence  position.  Finally, 
occultation  of  one  object  by  another  provides  an  angular  "line-of-positlon"  cue 
similar  to  one  component  of  a  navigation  "fix",  and,  hence,  also  provides  a 
means  of  inferring  position. 

The  cues  listed  in  Table  1  are  essentially  monocular  cues,  and  do  not 
depend  on  binocular  processing  capabilities.  In  addition,  these  cues  can  be 
provided  by  a  fairly  abstract  display:  a  perspective  line  drawing  of  the 
scene,  with  "hidden"  lines  not  drawn.  Since  the  basic  element  of  such  an 
idealized  display  is  a  line  segment  (associated  with  an  edge  of  solid  object), 
it  seems  appropriate  to  consider  in  more  detail  just  how  such  line  elements 
can  provide  cues  as  to  one's  own  attitude  and  position. 


Recent  OCM-Based  Cueing  Analyses 

To  see  how  these  cues  can  be  incorporated  within  the  structure  provided  by 
the  OCM,  it  is  appropriate  to  briefly  discuss  two  earlier  approaches. 

Wewerinke  (1978)  concentrated  on  the  pilot's  use  of  perspective  geometry 
and  motion  parallax  cues  during  a  VFR  landing-approach.  By  use  of  elementary 
perspective  geometry,  it  was  shown  that  rotational  and  translational  vehicle 
movements  can  be  related  to  changes  in  orientation  and  location  of  line  seg¬ 
ments  comprising  the  extra-cockpit  visual  scene.  For  example,  the  perturba¬ 
tions  in  the  perceived  orientation  of  the  runway  center  line,  with  respect 
to  the  aircraft's  longitudinal  plane  of  symmetry,  can  be  expressed  as: 

w  ■  C.h+  C  y  +  4> 
h  y 

where  w  is  the  center  line  orientation,  C.  and  C  are  trajectory  dependent 

h  y 
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constants  and  h,y,  and  <p  are  small  perturbations  in  altitude,  localizer  error, 
and  roll  attitude,  respectively.  Similar  expressions  can  be  derived  for  orienta¬ 
tional  changes  in  other  line  elements  comprising  the  visual  scene,  with  corres¬ 
ponding  expressions  for  their  rates  of  change. 

To  model  pilot  processing  of  these  cues,  Wewerinke  (1978)  used  the  OCM  and 
its  "display  vector"  capability:  each  w^,  associated  with  a  scene  line  element 

was  linearly  related  to  the  vehicle  state  vector  x  so  that 

wi  =  C^x  (8) 

where  is  the  vector  "display  gain"  associated  with  the  ith  line  element. 

If  the  pilot’s  display  vector  ^  is  composed  of  N  extra-cockpit  geometric  cues. 


then 

y  =  Cx 

(9a) 

where 

- =  ifirV-V1 

(9b) 

so  that  the  display  matrix  C,  introduced  earlier  in  (2) ,  is  completely  defined 
for  the  purposes  of  model  analysis. 


To  test  this  hypothesis  of  visual  cue  processing,  Wewerinke  conducted  a 
model  analysis  of  experimental  data  obtained  from  a  simulated  VFR  approach  and 
found  that  the  data  could  be  closely  matched  by  assuming:  a)  approximately 
"nominal"  pilot-related  model  parameters;  b)  optimal  attention  allocation 
between  display  vector  elements;  and  c)  individual  display  thresholds  which 
were  consistent  with  visual  perception  thresholds  found  in  related  psycho¬ 
physical  experiments.  In  short,  this  initial  modeling  approach  toward  visual 
cue  processing  was  well-supported  by  the  experimental  data. 

A  simpler  approach  was  utilized  by  Levison  and  Baron  (1976)  in  their  study 
of  cockpit  display  designs  for  a  terminal  configured  vehicle.  One  of  the 
displays  analyzed  included  a  perspective  drawing  of  the  runway.  The  OCM-based 
model  analysis  incorporated  this  information  by  assuming  that  runway  centerline 
orientation  was  an  element  of  the  pilot's  display  vector,  thus  providing  a 
direct  means  for  amalgamating  perspective  information  with  more  conventional 
"instrument"  information  in  a  logically  consistent  manner. 

These  studies  demonstrate  that  the  OCM  can  be  used  to  analyze  extra¬ 
cockpit  cueing,  if  two  basic  components  of  the  model  can  be  specified:  the 
C  matrix  of  the  "display"  equation  (2),  which  relates  the  vehicle  state  to  the 
cue  environment;  and  the  observation  noise  vector  v  of  the  "perceptual" 
equation  (3),  which  specifies  the  fidelity  of  the  e5ctra-cockpit  cues.  In  the 
following  sections  we  outline  a  general  approach  for  this  specification. 


Specification  of  Display  Matrix 


General  Approach 


Our  basic  objective  is  to  specify  how  linear  perturbations  in  vehicle 
state  x  give  rise  to  corresponding  perturbations  in  the  extra-cockpit  cue  set 
To  do  this,  we  first  define  an  appropriate  "total"  f  :ate  vector  X,  where 
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(10a) 


where  £  and  a  specify  the  vehicle's  position  and  attitude,  in  some  specified 
earth -fixed  coordinate  system.*  For  example,  if  we  choose  the  standard  north- 
east-down  Cartesian  frame,  then  £  and  a  may  be  given  by: 

L  =  (x,y,z)  T  ;  a  =  (ip,e,<p)T  (10b) 

where  £  is  defined  by  the  vehicle's  Cartesian  coordinates,  and  a  by  the  corres¬ 
ponding  Euler  angles. 


We  then  assume  that  we  can  define  a  functional  relationship  between  this 
total  state  vector  X  and  a  given  extra-cockpit  cue  Y^,  of  the  following  form: 

=  fi(X)  (11) 

where  f^  specifies  the  functional  relationship.  If  N  such  cues  comprise  a  cue 
set,  we  can  then  define  a  cue  vector  according  to 

YS  (Y1,Y2,...,Yn)T  (12) 

so  that 

I  =  f(X)  (13) 


where  components  of  f^  are  the  individual  f^. 

linear  vector  function,  but  the  corresponding 
rather  directly: 


6Y  = 


6X 


In  general  this  will  be  a  non¬ 
linear  relationship  can  be  found 

(14) 


Here,  5X  and  <5y  are  first-order  perturbations 
the  bracketed  term  is  the  sensitivity  matrix, 
Ofi/8Xj). 


in  the  state  and  cue  vectors,  and 
whose  (i,j)  component  is  given  by 


This  expression  is  simply  another  form  of  the  "display"  equation  (2),  if 
we  let  ran 

C  L3xJ  11  5  6Y  ;  x  =  6X  (15; 

Thus,  equation  sequence  (10)- (15)  provides  us  with  a  general  means  for  specify¬ 
ing  the  appropriate  C  matrix,  once  the  cue  set  Y  is  identified,  and  the 
functional  relationships  f  specified.  In  effect,  it  is  a  generalization  of  the 
approaches  cited  earlier  (Wewerinke,  1978;  Levison  and  Baron,  1976),  and  allows 
for  a  unified  treatment  of  the  many  cues  available  from  an  extra-cockpit  scene. 


Line  Element  Cueing 


We  will  not  go  into  detail  here  concerning  specific  visual  cue  set 
definitions,  but  it  is  appropriate  to  illustrate  the  approach  by  considering 
the  cues  available  from  a  single  line  element  in  the  visual  scene. 


In  general,  the  total  state  vector  would  also  include  higher  derivatives 
associated  with  vehicle  dynamics,  etc. ,  but  for  simplicity  of  discussion,  we 
neglect  them  here. 
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Each  segment  is  defined  by  two  end  points,  and  each  such  endpoint,  as 
viewed  by  the  observer,  has  a  unit  length  line-of-sight.  (LOS)  vector  associated 
with  it,  pointing  from  the  observer  to  the  endpoint.  With  two  three-dimensional 
LOS  vectors,  and  with  the  constraint  that  each  LOS  vector  is  unit  length,  it 
then  follows  that  a  visual  scene  element  has  associated  with  it  four  indepen¬ 
dent  parameters  which  specify  it,  relative  to  the  observer. 


Any  four-dimensionai  cue  set  which  contains  the  information  in  the  LOS 
vectors  will  suffice  for  a  display  analysis.  However,  from  a  perceptual  model¬ 
ing  standpoint,  it  is  clearly  more  appropriate  to  work  with  informational 
quantities  (cues)  whose  utility  can  be  justified  on  psychophysical  grounds.  An 
appropriate  set  would  appear  to  be  the  four  basic  stimulus  attributes  associated 
with  a  visual  scene  line  segment;  length,  orientation,  and  location  in  the 
visual  field,  with  location  reflecting  two  attributes  because  the  associated 
unit- length  LOS  vector  is  specified  by  two  independent  parameters  (e.g.,  azimuth 
and  eluvation  in  the  observer's  frame). 

Figure  2  illustrates  how  a  visual  scene  line  segment  P^P.  might  be  project¬ 
ed  on  an  observer-centered  unit  sphere.  The  two  LOS  vectors  u^  and  u.  define 
the  projection  endpoints  and  Pj  ,  and  also  serve  to  define  the  "length"  of 
the  element,  in  angular  units  subtended  at  the  observer's  location: 


E.  .  =  cos  x(u. *u.) 

v-i  ~z 


(16a) 


The  "location"  of  the  element  can  be  defined  via  the  LOS  vector  pointing  to 
the  visual  midpoint,  which  is  given  simply  by: 


-ij  = 


Note  that  this  is  a  coordinate-free  specification  of  location;  observer-refer¬ 
enced  "location"  coordinates  can  be  determined  once  an  appropriate  reference 
system  is  chosen.  For  example,  direction  cosines  specifying  the  midpoint  vector 
can  be  obtained  by  taking  the  dot  product  of  u^  with  the  observer's  three 

reterence  axes;  alternatively,  observer- referenced  azimuth  and  elevation 
coordinates,  specifying  midpoint  location,  can  be  obtained  by  the  appropriate 
geometric  transformation. 

Finally,  the  "orientation"  of  the  element,  measured  with  respect  to  the 
midpoint  meridian  and  identified  by  v,  in  the  figure,  can  be  specified  as 
follows:  ^ 


=  tan"  ((-uz*ui^)/(u2*uij) 


(16c) 


where  u  is  an  observer- fixed  reference  vector,  ana  uj".  and  u, ,  are  determined 
-z  -ij  -ij 


by  the  endpoint  LOS  vectors  according  to: 


-ij  *  (-i  x  -j^-i  X  -j! 

%j  ■  ^  •  V'lai  •  si' 


(17a) 


(17b) 
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Figure  2.  Spherical 
Projection  of  a  Line 
Segment 


This  functional  specification  of  the  line  element  cues,  given  by  (16a, b,c), 

corresponds  directly  to  the  general  relationship  given  by  (13)  once  we  specify 

the  dependence  of  the  LOS  vectors  u  and  _u,  ,  and  the  reference  vector  u  ,  on 

i  J  z 

vehicle  state  X.  This  is  done  directly  by  recognizing  that  if  vehicle  (observer) 
location  is  given  by  jr,  while  the  location  of  endpoint  P_L  is  given  by  r^,  then 

the  relative  location  of  P^  with  respect  to  the  observer  will  be  given  by 


C-i  =  Xi-!  (18a) 

so  that 

-i  =  fi^l&il  (18b) 


where  a  corresponding  relation  holds  for  u..  For  u  ,  i  can  be  shown  that  if 

J  z 

we  specify  the  observer's  orientation  vector  a  (comprised  of  the  three  Euler 
angles),  then  jj  can  be  expressed  as  an  explicit  vector  function  of  the  follow¬ 
ing  form: 

Xz  =  £(fl.)  (19) 


With  _r  and  ^  specifying  the  vehicle  state,  it  then  follows  that  (16)  through 
(19)  specify  the  desired  non-linear  cueing  relationship  summarized  earlier  by 
(13). 


With  this  cueing  relationship  so  specified,  it  is  now  a  direct  matter  of 
perturbation  analysis  to  derive  the  corresponding  linear  display  equations  sum¬ 
marized  by  (14).  We  will  not  do  this  for  the  entire  cue  set  just  described, 
but  simply  illustrate  the  approach  for  the  "length"  cue  £  . 


by 


From  (16a)  it  can  be  shown  that  the  change  in  length  can  be  expressed 


6^^  =  tjijj*  (6uj-6u  )  ]/cos(F^/2) 


i 
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where  u  was  defined  earlier  by  (17b)  and  6u. 

ij  — 1 


and  6uj  are  perturbations  in 


the  two  endpoint  LOS  vectors.  These,  however,  can  be  obtained  from  (18), 
assuming  that  the  vehicle  (observer)  undergoes  a  perturbation  in  position  <5_r, 
while  the  endpoint  locations  of  the  (physical)  line  element  remain  fixed  in 
space.  The  resulting  expression  for  iSu^  is  then  given  by: 

(21a) 
(21b) 


Siii  -  U±6  r 


where 


Ui  = 


where  I  is  the  identity  matrix  and  g,  is  defined  by  (18a).  A  corresponding 
relation  holds  for  6u^. 

These  expressions  for  and  5u^,  taken  in  tandem,  constitute  a  single 

scalar  relation  between  the  perturbation  in  the  length  cue,  and  the  positional 
change  in  vehicle  state.  Similar  expressions  can  be  found  for  perturbations  in 
cue  orientation  and  position,  although,  because  of  their  functional  dependence, 
they  involve  not  only  perturbations  in  vehicle  position  jr ,  but  also  orientation 
a.  The  resulting  set  of  four  linear  cueing  relations  can  thus  be  expressed  in 
the  matrix-vector  format  of  (14),  or,  equivalently,  in  the  display  equation 
format  of  (2). 


It  should  be  clear  that  this  type  of  analysis,  which  has  concentrated  on 
single-element  cueing,  can  be  extended  to  multi-element  cueing  to  account  for 
additional  cues  arising  from  geometric  relationships  between  different  elements 
(e.g.,  the  apparent  included  angle  between  two  line  segments  sharing  a  common 
visual  endpoint).  In  addition,  semi- infinite  segments  can  be  treated  as  a 
special  case,  to  allow  for  scene  elements  which  "disappear"  at  the  visual 
horizon  (e.g.,  parallel  infinite  lines). 


Specification  of  Observation  Noise 

The  second  component  of  the  perceptual  model  needing  specification  is  the 
observation  noise  vector  v  ,  associated  with  the  "perceptual"  equation  (3) . 

As  noted  earlier  in  the  OcK  description,  this  implies  a  specification  of  the 
thresholds  associated  with  each  cue,  and  the  allocation  of  attention  among 
members  of  the  entire  visual  cue  set. 

Visual  Thresholds 

Thresholds  can  be  specified  in  one  of  two  ways:  a  priori  on  the  basis  of 
related  psychophysical  measures,  or  a  posteriori  on  the  basis  of  "calibration" 
tracking  experiments. 

A  priori  specification  can  provide  a  first  approximation  to  the  relevant 
pilot-related  thresholds,  if  an  appropriate  data  base  exists.  Thus,  for 
example,  acuity  estimates  can  be  used  to  specify  a  "minimum"  threshold  for 
line  segment  orientation  (Levison,  1976).  If  data  is  lacking,  static  discrim¬ 
ination  experiments  can  be  designed  to  identify  thresholds  relevant  to  the 
specific  elements  comprising  the  extra-cockpit  scene  (Wewerinke,  1978).  In 
either  case,  such  estimates  usually  suffer  from  the  static  nature  of  the 


discrimination  task  being  used  to  identify  the  threshold,  and  the  lack  of 
correspondence  between  such  a  task,  and  the  task  for  which  the  cues  are  being 
used  (e.g.,  flying  an  airplane). 


These  shortcomings  provide  the  motivation  for  performing  "calibration" 
experiments,  from  which  relevant  thresholds  can  be  inferred.  To  illustrate 
the  approach,  we  briefly  describe  an  experimental  study  of  closed-loop  compen¬ 
satory  tracking,  using  for  a  display  an  electronically  driven  error  bar,  which 
moved  with  respect  to  a  stationary  reference  bar  (Levison,  1971).  Rate  (K/s) 
dynamics  were  used,  four  levels  of  display  gain  were  explored,  and  both  foveal 
and  parafoveal  ("peripheral")  viewing  conditions  were  considered.  Model  analysis 
via  the  OCM  yielded  estimates  of  observation  noise  for  both  displacement  and 
rate  perception  for  each  experimental  condition. 

Figure  3  shows  the  experimentally-derived  relationship  between  displace¬ 
ment  (rate)  observation  noise  and  mean-squared  display  deflection  (velocity). 
Results  are  shown  for  both  foveal  and  peripheral  viewing,  and  are  repeated  in 
Figure  4  on  an  expanded  scale  to  better  describe  the  relationships  at  the  low 
end  of  the  scale.  In  general,  noise  variance  varied  linearly  with  respect  to 
signal  variance,  in  accordance  with  the  noise  model  specified  earlier  by  (4). 


Figure  3.  Observation  Noise  Power 
Density  vs.  Signal 
Variance  (data/model 
comparison;  expanded  scale) 


Figure  4.  Observation  Noise  Power 
Density  vs.  Signal 
Variance  (data/model 
comparison) 


The  results  shown  in  the  figures  also  agree  both  qualitatively  and  quanti¬ 
tatively  with  other  results  found  in  the  literature  relating  to  perceptual 
thresholds.  Specifically: 

1.  Extrapolation  of  the  curves  to  zero  display  variance  reveals  a 
positive  intercept  on  the  observation  noise  axis  —  a  condition  that  must  hold 
if  these  results  are  to  reflect  a  physically  realizable  threshold-like  phenom¬ 
enon. 

2.  Threshold  values  derived  on  the  basis  of  the  zero  intercepts  are  of 
the  same  order  of  magnitude  as  threshold  values  reported  elsewhere  in  the 
literature. 

3.  Zero  intercepts  for  peripherally-viewed  quantities  are  greater  than 
intercepts  for  corresponding  quantities  viewed  foveally,  a  finding  which  is 
consistent  with  the  observation  that  higher  visual  thresholds  are  associated 
with  peripheral  viewing. 

4.  Peripheral-foveal  differences  are  proportionately  less  for  rate 
perception  than  for  displacement  perception,  a  result  consistent  with  the 
most  widely-held  belief  that  moving  objects  in  the  visual  periphery  are  "seen 
better"  than  stationary  objects. 

Because  of  the  reasonableness  of  these  results  (and  the  lack  of  contra¬ 
dictory  evidence),  the  inferred  thresholds  have  been  used  in  subsequent  studies 
involving  model  predictions  of  pilot/vehicle  performance  in  realistic  flight 
control  tasks,  with  considerable  success. 

A  similar  approach  to  threshold  determination  can  be  applied  to  the  extra¬ 
cockpit  cueing  environment.  What  is  required  is  a  simplified  pictorial  display, 
which  deliberately  limits  the  pilot's  information  base,  combined  with  a  well- 
defined  closed-loop  piloting  task.  OCM-based  model  analysis  of  the  resulting 
data  can  then  be  used  to  infer  appropriate  thresholds  (for  example,  for  line 
element  length,  orientation,  and  location  in  the  visual  field),  which  can  then 
be  used  to  extrapolate  to  situations  involving  richer  cue  environments. 

To  illustrate.  Figure  5  shows  a  perspective  display  of  an  idealized  roadway, 
which  might  be  used  to  study  lateral  path  regulation  of  an  aircraft  flying  at 
constant  altitude  over  flat  terrain.  The  display  can  provide  cues  for  infer¬ 
ring  both  position  and  attitude.  Position  cues  are  provided  by:  the  apparent 
angles  between  the  roadway  and  the  horizon  (a^  and  82),  which  reflect  lateral 

displacement  from  the  road  centerline;  and  the  apparent  path  angle  (b),  which 

reflects  altitude  above  the  road.  Attitude  cues  are  provided  by:  the  horizon 
line,  which  reflects  the  aircraft’s  roll  attitude;  and  the  vanishing  point 
location,  which  reflects  the  aircraft's  heading  with  respect  to  the  road 
centerline. 

The  control  task  of  minimizing  lateral  deviations  corresponds  to  the 
perceptual  task  of  determining  when  the  angles  a^  and  appear  to  be  equal. 

Likewise,  the  control  task  of  maintaining  wings  level  corresponds  to  the 
perceptual  task  of  determining  when  the  horizon  line  is  parallel  to  the  upper 
and  lower  frame  lines.  Thus,  corresponding  perceptual  thresholds  can  be 
determined  on  the  basis  of  data  obtained  from  an  appropriately-designed 
control  task,  and  with  the  use  of  the  OCM  to  infer  equivalent  perceptual 


thresholds  from  the  measured  response  data.  Clearly,  perceptual  thresholds 
associated  with  other  scene  elements  and  scene  element  attributes  can  be  deter¬ 
mined  in  a  similar  fashion. 


Figure  5.  Perspective  Display  of  Path 


Attentional  Allocation 

The  specification  of  the  observation  noise  vector  also  requires  a  determina¬ 
tion  of  the  attentional  allocation  among  members  of  the  total  visual  cue  set. 
Reference  to  (7)  shows  that  this  reduces  to  a  specification  of  the  individual 
fraction  of  attention,  f^,  associated  with  the  ith  cue,  for  all  cues  presented 
to  the  pilot. 

This  may  be  accomplished  simply  by  hypothesizing  a  "reasonable"  allocation 
strategy,  based  on  a  knowledge  of  the  informational  requirements  Imposed  by  the 
control/monitoring  task.  This  has  been  fairly  successful  in  past  applications, 
mainly  because  taak  performance  is  often  insensitive  to  variations  in  attention¬ 
al  allocation.  Alternatively,  one  may  build  upon  the  basic  optimality  assump¬ 
tion  of  the  OCM,  and  postulate  that  the  attention-sharing  strategy  be  determined 
so  as  to  optimize  performance  (Kleinman,1976) .  Solving  this  optimization 
problem  can  be  accomplished  in  a  straightforward  (although  expensive)  manner  by 
simply  adding  an  "outer"  optimization  loop  to  the  OCM,  which  iteratively  adjusts 
individual  attention  levels  until  task  performance  can  no  longer  be  significantly 
improved.  The  resulting  attention  levels  then  effectively  comprise  a  prescript¬ 
ive  specification  of  attentional  allocation,  for  "best"  task  performance  with  a 
given  cue  set. 
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Program  for  Experimental  Validation 

To  validate  this  model  of  extra-cockpit  visual  cueing  requires  a  well- 
defined  experimental  program,  focussed  on  the  identification  of  critical  pilot- 
related  model  parameters.  In  this  section  we  briefly  outline  a  three-phase 
program  which  should  be  capable  of  providing  the  type  of  validation  data  deemed 
necessary. 

The  task  of  lateral  path  regulation  at  constant  altitude  provides  a  con¬ 
venient  compromise  between  a  highly  realistic  task  (which  introduces  modelling 
uncertainties  due  to  task  complexity)  and  a  highly  idealized  task  (which  is 
quite  tractable  but  has  little  face  validity).  A  display  similar  to  the  one 
sketched  earlier  in  Figure  5  could  be  used  to  provide  the  pilot  with  the  needed 
position  and  attitude  cues:  position  from  the  apparent  inclination  of  the  road¬ 
way,  and  attitude  from  the  horizon  line  orientation  and  vanishing  point  location. 

By  artificially  constraining  the  task  dynamics,  a  series  of  single  axis 
sub-experiments  could  be  performed  to  quantify  the  pilot's  ability  to  utilize 
these  individual  cues.  For  example,  vehicle  position  could  remain  fixed  and 
centered  over  the  roadway,  and  the  pilot  given  the  task  of  single-axis  roll 
control.  By  varying  the  effective  display  gain  (the  ratio  of  scene  rotation 
angle  to  vehicle  roll  angle),  the  roll  axis  threshold  could  be  determined,  in  a 
manner  analogous  to  that  discussed  earlier  and  illustrated  in  Figures  3  and  4. 

In  a  similar  fashion,  roll  attitude  and  heading  could  be  constrained,  and  the 
pilot  given  the  task  of  simple  lateral  path  control.  As  noted  earlier,  this 
would  provide  a  means  of  inferring  the  effective  threshold  associated  with  the 
(Imaginary)  roadway  centerline  orientation,  again  assuming  that  display  gain 
variation  is  used  as  an  experimental  variable.  Thresholds  associated  with 
other  scene  elements  could  be  determined  in  a  similar  fashion. 

Once  these  thresholds  had  been  determined,  the  full  multi-axis  experiment 
could  be  conducted,  with  the  aim  of  verifying  the  attention-sharing  predictions 
generated  by  the  OCM.  Various  hypotheses  could  be  investigated  via  pre-experi- 
mental  model  simulations  (e.g. ,  attentional  optimization,  full  cue  interference 
due  to  scanning  requirements,  minimal  cue  interference  due  to  the  "integrated" 
nature  of  the  display,  etc.),  and  the  corresponding  model  predictions  of  pilot 
performance  compared  to  the  measured  data.  The  "best"  match  between  model  and 
data  would  then  identify  the  model  of  perceptual  processing  most  likely  used  by 
the  pilot. 

A  second  phase  of  the  validation  program  could  concentrate  on  the  pilot's 
capabilities  for  altitude  control,  using  the  same  display  as  in  the  first 
phase.  Here,  the  primary  height  cue  would  be  the  apparent  central  angle  defined 
by  the  roadway  edges,  while  the  primary  flight  path  angle  cue  would  be  provided 
by  the  depression  angle  of  the  horizon  line.  Again,  sub-experiments  could  be 
used  to  identify  individual  cue  thresholds  (via  display  gain  variations),  and 
a  combined  multi-axis  experiment  could  be  conducted  to  validate  model  predictions 
of  attentional  allocation  among  cues  providing  vertical  path  control  information. 

The  third  and  final  phase  of  the  validation  program  could  combine  both 
lateral  and  vertical  tasks,  in  a  full  five  degree-of-freedom  flight  control  task 
(at  constant  airspeed).  The  primary  objective  of  this  phase  would  be  the  valida¬ 
tion  of  model  predictions  of  attention  allocation,  between  lateral  and  vertical 
axes,  and  among  cues  associated  with  each  axis. 
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Determination  of  Simulator  Visual  Cueing  Requirements 

The  discussion  to  this  point  has  been  concerned  primarily  with  the 
perceptual  limitations  associated  with  the  pilot,  their  impact  on  overall  task 
performance,  and  their  efficient  representation  within  the  structure  provided  by 
the  OCM.  However,  it  should  be  pointed  out  that  the  model,  as  defined,  makes  no 
explicit  distinction  between  pilot-related  perceptual  limitations  and  simulator- 
related  cueing  limitations.  Thus,  the  model  is  as  applicable  to  a  study  of 
simulator  cueing  requirements,  and  their  impact  on  closed-loop  pilot  performance, 
as  it  is  to  the  above-discussed  studies  of  inherent  human  perceptual  limitations. 

To  use  the  model  in  this  manner  requires  an  initial  translation  step,  from 
the  basic  simulator  cueing  characteristics  (actual  or  proposed),  to  the  under¬ 
lying  model  parameters  which  characterize  the  perceptual  "front-end"  of  the  OCM. 
An  illustration  of  this  mapping  process  is  given  in  Figure  6,  which  is  taken  from 
an  analytic  study  of  simulator  cueing  requirements  for  helicopter  hover  (Baron 
et  al,  1980).  Parameters  associated  with  the  computer-generated  imaging  (CGI) 
system  were  "mapped"  into  equivalent  OCM  parameters  (e.g.,  the  CGI  display  delay 
value  was  used  to  adjust  the  perceptual  delay  associated  with  the  OCM's  visual 
channel),  so  that  the  resulting  OCM  parameters  reflected  not  only  the  basic 
human  limitations,  but  also  the  non- ideal  characteristics  of  the  simulator. 
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Figure  6.  Simulator  Mappings  for  OCM  Analysis 
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Once  this  mapping  is  accomplished,  the  OCM  may  then  be  used  to  generate  perform¬ 
ance  predictions,  for  the  human  pilot  operating  in  the  specified  simulator  cueing 
environment.  If  this  mapping  is  done  for  a  variety  of  configurations,  and  over 
a  reasonable  range  of  simulator-associated  parametric  values,  corresponding 
performance  predictions  can  then  be  "swept-out",  by  repeated  use  of  the  OCM. 

This  was  done  in  the  cited  study  (Baron  et  al,  1980),  and  Figure  7 
illustrates  the  type  of  closed-loop  sensitivity  predictions  which  can  be  genera¬ 
ted  by  this  approach.  The  ordinate  represents  the  relative  hover  performance 
decrement  in  percent:  J  is  a  composite  scalar  metric  reflecting  overall  hover 

accuracy  in  the  simulator,  and  J_  „  is  the  corresponding  metric  associated  with 

r  LI 

the  actual  flight  situation.*  Since  the  ordinate  is  the  normalized  difference 
between  these  two  metrics,  it  is,  in  effect,  a  fidelity  metric  for  comparing 
in-simulator  performance  to  in-flight  performance,  with  a  zero  value  indicating 
perfect  simulator  fidelity.  The  abcissa  represents  time  delay  in  the  CGI 
computer,  and  the  curves  (which  are  parameterized  against  mainframe  computer 
cycle  time)  show  how  rapidly  performance  fidelity  degrades  with  increasing  delay. 

Similar  fidelity  curves  can  be  generated  for  other  simulator  cueing  charac¬ 
teristics  (e.g.,  CGI  resolution,  field-of-view,  etc.),  to  support  an  extensive 
specification  of  the  relationship  between  cueing  deficiencies  and  expected 
simulator/aircraft  performance  mismatches.  One  can  readily  envision  how  such  an 
information  base  could  be  used  in  a  rational  specification  of  task-specific 
simulator  cueing  requirements.  Starting  with  a  desired  fidelity  level  to  be  met 
by  the  simulator,  one  could  refer  to  the  curves  to  identify  preliminary  parameter 
ranges  which  ensure  that  this  level  will  be  met  or  exceeded.  Engineering  and 
cost  trade-offs  could  then  be  made  within  these  ranges  (e.g.,  balancing  the 
increase  in  simulator  fidelity  achieved  by  increasing  the  CGI  resolution,  against 
the  longer  development  times  and  higher  costs  associated  with  better  quality 
hardware) ,  to  arrive  at  a  reasonable  choice  of  simulator  specifications  which 
will  ensure  meeting  the  desired  fidelity  level. 

More  sophisticated  procedures  can  also  be  envisioned,  which  take  into 
account  cross-coupling  effects  between  simulator  parameters,  to  provide  the 
designer  with  a  "hyper-ellipse”  of  acceptable  simulator  parameter  choices,  given 
a  desired  fidelity  level.  It  is  not  our  intent  to  speculate  on  possible  advanced 
design  algorithms,  however,  but  rather  to  point  out  potential  applications  of 
model-based  fidelity  metrics.  It  would  seem  that  this  general  approach  holds 
considerable  promise  for  the  rational  and  successful  specification  of  simulator 
cueing  requirements . 


*Ihe  latter  was  calculated  simply  by  assuming  an  ideal  simulator  having  no  cueing 
deficiencies,  so  that  it  replicated  the  flight  environment  perfectly. 
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RELATIVE  PERFORMANCE  I— j - I  M% 


wr 


Figure  7a.  Relative  Performance  Vs. 
Time  Delay  (Longitudinal  Axis  - 
Fixed  Base) 


Figure  7b.  Relative  Performance  Vs. 
Time  Delay  (Lateral  Axis  - 
Fixed  Base) 


Summary 

In  this  paper  we  have  outlined  an  analytic  and  experimental  effort  aimed 
at  developing  and  validating  a  quantitative  model  of  extra-cockpit  visual  cueing. 
The  analytic  approach  uses  the  basic  "perceptual"  sub-model  of  the  OCM,  and 
concentrates  on  how  such  cues  relate  to  the  basic  aircraft  flight  control 
variables  of  interest  to  the  pilot.  The  analysis  was  limited  to  those  cues  pro¬ 
vided  by  perspective  geometric  scenes  of  the  extra-cockpit  world,  and  a  general 
geometric  and  perturbation  analysis  of  the  simulator  environment  was  discussed. 
Its  use  in  identifying  the  appropriate  OCM-related  "display  equation"  was  illus¬ 
trated  by  concentrating  on  single  line  element  cues,  and  their  informational 
content. 

The  experimental  approach  concentrates  on  identifying  pilot-associated 
perceptual  limitations  in  processing  the  available  visual  cues.  A  general 
flight  task  and  extra-cockpit  display  was  proposed  for  use  both  in  the  identifi¬ 
cation  of  relevant  visual  thresholds,  and  in  the  determination  of  human 
attention  allocation  strategies.  Thresholds  are  determined  by  the  use  of 
appropriate  sub-tasks  combined  with  variations  in  display  sensitivity,  while 
attention  allocation  strategies  are  identified  by  hypothesis  testing  and  model 
matching. 
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Finally,  a  general  approach  for  the  rational  specification  of  simulator 
cueing  requirements  was  outlined.  It  is  predicated  on  a  validated  OCM-based 
model  of  extra-cockpit  cueing,  and  introduces  the  notion  of  a  model-based  simula¬ 
tion  fidelity  metric.  The  approach  provides  a  means  of  specifying  desired 
fidelity  levels,  and  identifying  appropriate  simulator  configurations  and 
parameter  ranges,  as  a  starting  point  for  further  engineering  and  cost  trade-off 
studies  in  the  simulator  specification  design  cycle. 
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TRANSFORMATION  REALISM: 

AN  INTERACTIVE  EVALUATION  OF  OPTICAL  INFORMATION 
NECESSARY  FOR  THE  VISUAL  SIMULATION  OF  FLIGHT 


Our  research  is  an  attempt  to  (1)  isolate  optical 
variables  and  invariants  useful  in  the  perception  of  self 
motion j  and  (2)  use  performance  measia^es  to  determine  the 
information  used  in  guiding  flight.  Initial  studies  have 
been  concerned  with  the  role  of  optical  flow  and  density 
parameters  in  sensitivity  to  change  in  speed  and  altitude. 

Advances  in  hardware  and  graphics  technique  continuously  result  in  more 
realistically  detailed  visual  simulation  of  environmental  surfaces.  The  goal 
of  surface  detail  realism  (e.g.,  trees,  buildings,  mountains)  is  pursued  with 
little  regard  for  either  its  necessity  or  the  possibility  that  attention  to 
irrelevant  information  might  impede  learning.  Our  approach  addresses  the 
problem  of  determining  the  optimal  information  in  kind,  quality,  and  magnitude 
that  must  be  made  available  in  order  to  produce  time-  and  cost-efficient 
transfer  from  simulation  to  actual  flight.  The  programmatic  phases  will  be 
described,  including  underlying  theoretical  assumptions  and  their  relation  to 
the  developing  methodology.  With  fairly  firm  grounding  on  the  principle  of 
visual  capture  or  dominance,  initially  only  visual  contact  sources  of  informa¬ 
tion  will  be  considered,  with  the  realization  that  integration  and  contrast 
with  vestibular,  haptic,  somatic,  and  coded  information  must  eventually  be 
considered . 

Our  approach  follows  Gibson  (e.g.,  1966,  1979)  in  distinguishing  between 
the  visual  information  specifying  environmental  surfaces  and  that  specifying 
the  relation  of  the  perceiving  individual  to  the  surfaces.  We  are  testing 
the  assumption  that  schematic  surface  detail  is  sufficient  for  learning,  if 
ecologically  valid  optical  transformations  are  available  to  specify  self 
motion.  The  initial  phase  of  our  program  has  been  concerned  with  the  identi¬ 
fication  of  potential  sources  of  optical  information,  including  ways  to 
specify  the  sources  mathematically  and  isolate  them  empirically.  The  primary 
focus  has  been  on  variables  of  optical  flow  and  optical  texture,  and  their 
usefulness  in  the  detection  of  change  in  altitude  and  speed. 

An  important  concept  for  exploring  the  relationship  between  environmental 
variables  (e.g.,  surface  texture  size,  shape,  and  color;  altitude  and  descent 
rate;  forward  speed  and  deceleration  rate)  and  optical  variables  is  that  of 
the  functional  invariant.  It  is  hypothesized  that  when  optical  conditions  are 
the  same,  regardless  of  the  environmental  variable  ratios  which  produced  them, 
performance  will  be  the  same.  The  hypothesis  is  based  on  the  assumption  that 
the  perceptual  system  only  has  access  to  optical  information,  and  that  ade¬ 
quate  knowledge  of  the  relation  between  self  and  environmental  surfaces  is 
available  to  guide  one's  own  locomotion. 

The  first  experimental  step  was  concerned  with  understanding  the  relation 
between  environmental  variables  and  both  optical  variables  and  invariants. 

Two  studies,  to  be  described  subsequently,  were  conducted  to  assess  sensiti¬ 
vity  to  constant  rates  of  loss  in  altitude  and  forward  speed.  The  results 
converged  on  a  common  kind  of  functional  invariant;  fractional  increase  or 
decrease  in  optical  rates  of  change.  The  second  step  consists  of  factorially 
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crossing  optical  variables  while  letting  environmental  variables  take  on  any 
values  necessary  to  simulate  a  desired  event.  This  step,  currently  under  way, 
allows  exploration  of  two  problems:  (1)  determining  the  appropriate  metric  for 
scaling  optical  invariants,  and  (2)  separating  speed-relevant  variables  from 
altitude-relevant  variables.  The  metric  issue  is  under  study  with  two  alter¬ 
native  candidates,  in  ground  scaled  units  (e.g.,  texture  block  size)  versus 
body  scaled  units  (e.g.,  eyeheights).  The  criterion  for  choice  of  metric  is 
whether  one  of  the  two  alternatives  results  in  functional  invariance  when  they 
are  factorially  crossed,  whereas  the  other  does  not.  Solution  of  this  problem 
is  critical  in  determining  what  an  individual  needs  to  know  in  order  to  per¬ 
ceive  distance  (including  altitude)  and  change  in  distance  veridically  (see 
Harker  &  Jones,  1980,  pp.  25-31). 

Change  in  speed  is  potentially  specified  by  increase  or  decrease  in  opti¬ 
cal  flow  rate  (Warren,  1981a).  Loss  in  altitude  is  potentially  specified  by 
several  optical  variables,  including  optical  flow  acceleration,  decrease  in 
optical  density,  and  increase  in  optical  "splay"  rate  (Warren,  1981b).  (Splay 
indexes  optical  discontinuities  along  lines  projecting  to  the  vanishing 
point.)  By  controlling  environmental  variables,  various  kinds  of  optical 
information  can  be  eliminated  in  order  to  assess  the  salience  of  those  re¬ 
maining.  For  example,  optical  acceleration  can  be  eliminated  during  descent 
by  decelerating  path  speed  exponentially  at  the  rate  required  to  hold  optical 
flow  rate  constant.  A  preliminary  experiment  indicates  that  descent  is  easily 
detected  on  the  basis  of  optical  density  and  splay-rate  changes.  Splay  rate 
can,  in  turn,  be  eliminated  as  a  source  of  information  by  using  surface  tex¬ 
ture  strips  with  edges  perpendicular  to  the  direction  of  travel  (i.e., 
parallel  to  the  horizon).  Optical  magnification  remains  as  a  source  of  infor¬ 
mation  for  descent,  although  preliminary  tests  suggest  that  observers  are  not 
as  sensitive  to  it  alone  as  to  combined  splay  and  density  changes.  In  addi¬ 
tion  to  those  examples  of  information  deletion  and  accretion,  sources  of 
information  can  be  made  irrelevant  in  contrasts  with  constant  velocity  or 
constant  altitude  trials,  or  held  constant  in  factorial  designs.  All  these 
types  of  control  should  prove  useful  in  isolating  the  optical  information 
necessary  for  simulation  of  a  given  flight  maneuver.  Further  work  will  be 
concerned  with  optimizing  these  variables  during  training. 

Given  that  tests  of  accuracy  and  efficiency  in  detecting  change  and  non¬ 
change  isolate  optical  invariants  in  passive  judgment  experiments,  the  next 
phase  casts  the  problem  in  a  research  framework  which  mimics  the  interactive 
nature  of  actual  flight.  The  experimenter  is  in  control  of  the  environmental 
variables  studied  in  the  first  phase,  namely  surface  texture  size,  shape,  and 
color,  and  the  variables  used  to  initiate  a  trial,  e.g.,  forward  velocity, 
altitude,  descent  rate,  and/or  deceleration  rate.  A  trial  is  initiated  by 
requiring  the  pilot  to  take  over  the  controls  and,  with  all  instruments 
covered,  assess  via  visual  contact  whether  there  is  a  deviation  from  the 
desired  flight  path.  If  so,  a  compensatory  control  adjustment  is  made. 
Efficiency  and  accuracy  of  the  control  actions  are  related  to  initial  optical 
values,  the  state  of  optical  stimulation  at  the  time  the  adjustment  was  made, 
and  a  ratio  of  the  change  in  stimulation  to  the  initial  optical  information 
values.  The  goal  of  this  analysis  is  to  determine  the  optical  conditions 
which  result  in  the  initiation  of  ad justments ,  lead  to  their  modification,  and 
ultimately  are  achieved  as  a  desired  situation  by  the  pilot. 
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The  interactive  paradigm  thus  turns  control  over  changes  in  stimulation 
to  the  pilot  as  occurs  outside  the  laboratory.  The  control  actions  become 
perceptual  reports ,  and  the  x,  y,  z  position  of  the  aircraft  over  time  can  be 
used  to  compute  optical  variables.  The  researcher's  task  is  to  perform  retro¬ 
spective  analyses  of  the  optical  information  available  to  determine  what  the 
pilot  used  to  guide  his  locomotion.  This  evidence  can  then  be  used  to  opti¬ 
mally  allocate  available  detail  (edges  and/or  points)  in  order  to  best 
simulate  any  given  maneuver. 

Success  of  this  phase  will  lead  to  the  study  of  complete  flight  subtasks, 
such  as  approach  and  landing  or  the  avoidance  of  obstacles,  and  eventually  to 
the  study  of  transfer  of  training  from  simulator  to  aircraft. 


Optical  Information  for  Detecting  Loss  in  One's  Own  Forward  Speed 

Warren  (1981a)  has  systematized  and  mathematically  isolated  candidates 
for  optical  information  specifying  relative  change  in  speed  during  self  motion 
(see  Appendix  A).  The  experimental  effort  described  here  is  an  initial 
attempt  to  explore  the  efficacy  of  this  approach.  Loss  in  speed  was  chosen 
because  it  occurs  in  every  case  of  locomotion,  and  appropriate  sensitivity  is 
necessary  to  guide  approach  to  surfaces.  When  separated  from  change  in  alti¬ 
tude  and  direction,  loss  in  speed  provides  an  optically  simpler  point  of 
departure  for  psychophysical  study  than  would  such  changes  in  the  path  of 
locomotion  itself. 

Given  a  constant  eyeheight  (altitude),  optical  deceleration  is  specific 
to  deceleration  in  the  velocity  of  movement  of  the  eye  above  the  ground  sur¬ 
face.  For  conditions  having  the  same  deceleration  in  rate  of  movement  over 
the  same  surface  texture  at  different  altitudes,  optical  deceleration  will 
differ  since  optical  flow  varies  with  eyeheight.  Lastly,  given  constant 
deceleration,  initial  forward  velocity  should  affect  sensitivity  to  decelera¬ 
tion  because  a  given  loss  in  optical  flow  rate  should  be  easier  to  detect 
relative  to  a  lower  than  to  a  higher  optical  flow  rate. 

Accordingly,  levels  of  deceleration  rate,  altitude,  and  initial  forward 
velocity  were  chosen  to  explore  the  ability  of  observers  to  distinguish  loss 
in  forward  speed  from  constant,  level,  forward  locomotion.  Levels  of  these 
three  environmental  independent  variables  were  chosen  so  as  to  produce 
matrices  with  five  levels  of  the  optical  flow  variables.  All  adjacent  envi¬ 
ronmental  variable  levels  have  the  same  ratio,  so  that  the  negative  diagonals 
of  the  two-factor  matrices  have  invariant  optical  flow  values.  The  design 
simultaneously  allows  assessment  of  the  effect  of  the  optical  variable  and 
determination  of  whether  an  optical  invariant  is  a  functional  invariant. 

Method.  A  special  purpose  visual  scene  generator  was  used  to  produce 
10-sec  sequences  representing  self  motion  over  a  flat  surface  covered  with 
rectilinear  texture  blocks.  The  center  strip  extending  toward  the  horizon 
was  fixed  at  10  m  in  width,  with  two  adjacent  strips  also  of  10  m.  This  30  m- 
wide  area  was  flanked  on  each  side  by  two  strips  of  30  m,  for  a  total  of  90  m. 
Successive  adjacent  strips  on  each  side  were  90  m  and  then  270  m,  so  that  the 
multiplier  of  three  was  used  throughout.  The  edges  perpendicular  to  the 
direction  of  travel  and  within  the  three  10  m-wide  strips  were  chosen  to 
produce  texture  blocks  from  5  to  20  ra  in  length.  Lengths  were  sampled 
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randomly  from  a  rectangular  distribution  having  lengths  in  all  5  m  steps  be¬ 
tween  these  extremes.  The  texture  blocks  were  filled  in  four  earth  colors: 
light  green,  dark  green,  tan,  and  a  darker  brown.  Colors  were  randomly 
assigned  to  the  blocks,  resulting  in  various  rectilinear  shapes  and  sizes. 

An  analog  computer  was  programmed  to  control  the  scene  generator  in  order 
to  vary  altitude,  initial  forward  velocity,  and  deceleration  rate.  The  10-sec 
trial  sequences  were  videotaped  and  displayed  on  a  Sony  video  projection 
screen  150  cm  wide  and  112.5  cm  high,  resulting  in  a  field  of  view  of  34  deg 
by  26  deg.  The  observer  sat  in  a  Link  General  Aviation  Trainer-1  flight 
simulator  with  the  motion  base  off.  His  point  of  observation  was  3.2m 
perpendicular  to  the  center  of  the  screen  at  a  height  of  1.67  m  which  matched 
the  fixed  position  horizon  on  the  screen.  All  sequences  represented  forward 
level  movement  over  the  center  of  the  surface  as  shown  in  Figure  1. 

3 

A  3  full-factorial  combination  of  deceleration  rate  (2,  3.5,  6.125 
m/sec/sec),  altitude  (18,  31.5,  55.125  m),  and  initial  forward  velocity 
(18,  31.5,  55.125  m/sec)  produced  27  different  deceleration  sequences.  The 
three  optical-variable  matrices  produced  by  these  combinations  are  shown  in 
Table  1.  Figure  2  shows  how  combinations  of  environmental  variable  levels 
produce  optical  flow  invariants,  in  this  case  for  optical  flow  damping. 
Combinations  of  altitude  and  forward  velocity  levels  produced  nine  different 
sequences  representing  forward  movement  at  constant  velocity  and  altitude. 

To  equal  the  number  of  deceleration  trials,  each  constant-velocity  sequence 
was  presented  three  times.  Each  observer  was  tested  with  three  different 
orders  of  the  54  sequences. 

The  observer  was  instructed  to  indicate  whether  the  event  displayed 
represented  movement  at  a  constant  speed  or  loss  in  speed  by  pressing  one  of 
two  appropriately  designated  buttons.  A  1000  Hertz  tone  initiated  .5  sec 
before  the  onset  of  the  trial  served  as  a  ready  signal.  Simultaneously  with 
the  beginning,  of  the  trial,  a  millisecond  timer  was  started.  Timing  termina¬ 
ted  when  either  button  was  pressed.  The  observer  was  unaware  that  time  to 
respond  was  being  recorded.  Following  the  button  press,  the  observer  rated 
his  confix  ince  in  the  choice  made  on  a  three-point  scale.  "Three"  represented 
very  certain;  "two,"  fairly  certain;  and  "one,"  guessing. 

Forty-two  male  volunteers  participated  in  partial  fulfillment  of  an 
introductory  psychology  course  requirement.  Each  answered  a  request  for  non¬ 
pilots  with  no  experience  in  flight  simulators. 

Results.  Analysis  of  the  errors  (from  reporting  "constant  speed"  on 
deceleration  trials)  revealed  no  significant  effect  for  sessions.  Mean  reac¬ 
tion  time  dropped  significantly  between  the  first  and  second  sessions,  but  the 
effect  was  small  (n2  =  3%).  Proportion  errors  decreased  with  an  increase  in 
deceleration  rate  (n2“12%)  and  increased  with  an  increase  in  forward  velocity 
(n2  =  7.8%).  The  interaction  of  these  two  variables  accounted  for  3.6%  of  the 
variance.  These  effects  are  shown  by  the  broken  lines  in  Figure  3A. 

Figure  3B  reveals  the  mean  error-free  reaction  time  decreased  with  increase  in 
deceleration  rate  (n2  =  21.5)  and  increased  with  increase  in  forward  velocity 
(n2  =  10.2%). 

Performance  was  next  evaluated  as  a  function  of  the  three  global  optical 
variables  computed  as  ratios  of  the  three  environmental  variables.  In  order 
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Table  1 


Combinations  of  Environmental  Variables  to 
Produce  Optical  Variables  and  Invariants 
A.  Initial  Global  Optical  Flow  Rate  (eyeheights/sec) 

Initial  Forward  Velocity  (m /sec) 


Altitude  (m) 

18 

31,5 

55.125 

20 

.900 

1.575 

2.756 

35 

.514 

.900 

1.575 

61.25 

.294 

.514 

.900 

B.  Global  Optical  Flow 

Deceleration  (eyeheights/s ec/sec) 

Deceleration  Rate 

(m/sec/sec) 

Altitude  (m) 

2 

3.5 

6.125 

20 

.100 

.175 

.306 

35 

.057 

.100 

.175 

61.25 

.033 

.057 

.100 

C.  Global  Optical  Flow  Damp ing  (%  eyeheight/sec) 


Initial  Forward  Deceleration  Rate  (m/sec/sec) 

i 


Velocity  (m/sec) 


2 


3.5 


6. 


.70 
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Velocity  Damping 

(m/sec )  (%  eyeheight  /sec) 

.  55.125  o  3.6 


0 


DECELERATION  RATE  (m  /sec/sec) 


Figure  3.  Mean  proportion  errors  (A)  and  mean  error-free  reaction 
time  (B)  as  a  function  of  deceleration  rate,  forward  velocity,  and  their 
ratio  (global  optical  flow  damping). 
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for  an  optical  invariant  to  be  considered  a  functional  invariant,  two  criteria 
must  be  met:  (1)  Performance  must  be  invariant  even  though  the  environmental 
variables  producing  the  optical  invariant  vary  over  a  broad  range.  (2)  Per¬ 
formance  must  vary  in  an  orderly  fashion  when  the  optical  invariant  has 
different  values.  The  latter  criterion  is  necessary  to  eliminate  cases  in 
which  performance  is  invariant  because  detection  is  too  easy  (ceiling  ef¬ 
fects),  too  difficult  (floor  effects),  or  the  optical  variable  is  unrelated 
to  the  particular  task  used. 

Using  these  two  criteria  to  identify  a  functional  invariant  indicated 
that  optical  flow  damping  was  the  best  candidate  for  invariant  information 
underlying  sensitivity  to  loss  in  forward  speed  of  locomotion.  The  solid 
lines  in  Figures  3A  and  3B  reveal  the  extent  to  which  optical  damping  invari¬ 
ants  resulted  in  constant  levels  of  performance. 


Optical  Information  for  Detecting  Loss 
in  One's  Own  Altitude 

Any  change  in  orientation,  direction,  or  speed  during  locomotion  may 
result  in  an  increase  or  a  decrease  in  an  optical  variable  and  may  also  leave 
some  optical  variables  unchanged.  Warren  (1981b)  has  systematized  and  mathe¬ 
matically  isolated  candidates  for  global  optical  information  specifying  rela¬ 
tive  change  altitude  during  rectilinear  self  motion  (see  Appendix  A).  The 
second  experiment  applied  the  paradigm  used  for  deceleration  to  the  optically 
more  complex  analysis  of  potential  functional  invariants  for  sensitivity  to 
loss  in  altitude.  Accordingly,  levels  of  descent  rate,  forward  velocity,  and 
initial  altitude  were  chosen  to  test  the  ability  of  observers  to  distinguish 
loss  in  altitude  from  level,  forward  locomotion.  Except  where  noted,  all 
conditions  were  the  same  as  for  the  first  experiment. 

Method.  Environmental  variable  levels  chosen  were  initial  altitudes  of 
20,  40,  and  80  m;  forward  velocities  of  18,  36,  and  72  m/sec;  and  descent 
rates  of  1.25,  2.50,  and  5.00  m/sec.  A  full-factorial  combination  of  these 
levels  produced  27  different  descent  sequences  and  nine  different  sequences 
representing  forward  locomotion  at  a  constant  altitude  that  were  repeated 
three  times. 

Table  2  shows  two  higher-order  environmental  variables  that  specify  the 
change  in  optical  locus  over  time:  glide  path  in  degrees  and  descent  rate  as 
a  percent  of  initial  altitude  per  second.  Table  3  shows  two  optical  flow 
variables:  optical  flow  rate  and  optical  flow  acceleration.  Figure  4  shows 
how  combinations  of  environmental  variable  levels  produce  optical  flow  invari¬ 
ants,  in  this  case  for  variables  specifying  initial  fractional  loss  in 
altitude. 

The  observer  was  instructed  to  indicate  whether  the  event  displayed 
represented  descent  or  level  movement  over  the  surface  by  pressing  one  of  two 
appropriately  designated  buttons.  The  54  trials  of  the  single  session  were 
randomized  individually  for  each  observer  with  the  constraint  that  mo  more 
than  four  level  or  four  descent  trials  would  occur  sequentially.  Twenty  male 
volunteers  participated  in  partial  fulfillment  of  an  introductory  psychology 
course  requirement. 
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Table  2 

Higher-Order  Environmental  Variables  and  Invariants 


Descent 

Rate  (m/sec) 

A. 

18 

Glide  Angle  (deg) 

Forward  Velocity  (m/sec) 

36 

72 

1.25 

3.97 

1.99 

1.00 

2.50 

7.91 

3.97 

1.99 

5.00 

15.52 

7.91 

3.97 

B.  Descent  Rate 

as  Percent  of  Initial  Altitude  (% 

eyeheight/sec) 

Initial 

Altitude  (m) 

1.25 

Descent  Rate  (m/sec) 

2.50 

5.00 

20 

6.25 

12.50 

25.00 

40 

3.13 

6.25 

12.50 

80 

1.56 

3.13 

6.25 

Table  3 


Global  Optical  Variables  and  Invariants 


A.  Initial  Global  Optical  Flow  Rate  (eyeheights/sec) 


Initial 

Altitude  (m) 

18 

Forward  Velocity 

36 

(m/sec) 

72 

20 

.900 

1.800 

3.600 

40 

.450 

.900 

1.800 

80 

.225 

.450 

.900 

3.  Initial  Global  Optical  Flow  Acceleration  (X  eyeheight/ 

sec/sec) 

Forward  Velocity 

Descent 

Rate  (%  eyeheight/sec) 

(eyeheights/sec) 

1.56 

3.13 

6.25 

12.50 

25.00 

.225 

.35 

.70 

1.41 

* 

* 

.450 

.70 

1.41 

2.81 

5.63 

* 

.900 

1.41 

2.81 

5.63 

11.25 

22.50 

1.800 

* 

5.63 

11.25 

22.50 

45.00 

3.600 

* 

* 

22.50 

45.00 

90.00 

*  These  cells  were  not  used  in  the  experiment. 
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INITIAL  RELATIVE  RATE  OF  CHANGE  (%/sec 


DESCENT  RATE  (m/sec) 

Figure  4.  Relative  rate  of  optical  change  at  the  initiation  of  a 
trial.  Solid  lines  show  optical  invariants;  broken  lines  are  for  dif¬ 
ferent  initial  altitudes. 


2 

Results.  For  errors,  significant  effects  included  altitude  (n  =  2.2%), 
descent  rate  (n2  =  3.9%),  and  altitude  by  descent  rate  (n2  =  1.8%).  For  reac¬ 
tion  times  significant  effects  included  altitude  (rp  *  13.1%),  forward 
velocity  (n2  *  .3%),  and  the  three-way  interaction  (n2  =  1.6%).  Figures  5  and 
6  show  effects  for  initial  altitude  and  descent  rate  (broken  lines),  as 
well  as  for  fractional  rate  of  change  in  altitude  (solid  lines).  Optical 
parameters  specific  to  fractional  change  are  functionally  invariant  for  all 
reaction  times,  but  only  for  low  error  rates. 


Discussion 

Of  the  candidates  considered  in  the  two  experiments,  relative  or  frac¬ 
tional  rates  of  change  best  meet  the  criteria  defining  a  functional  invariant 
for  detecting  change  in  one's  own  motion.  Simple  rates  of  optical  change 
(optical  flow  deceleration  and  acceleration,  increase  in  optical  density,  or 
change  in  splay  angle)  have  their  effects  perceptually  with  regard  to  their 
contribution  to  relative  rates  of  change. 

A  phase  analysis  of  the  relation  between  perception  and  the  environmental 
event  under  study  will  explicate  the  relationship  among  these  variables.  Three 
phases  relevant  to  egomotion  stimulation  will  be  considered  here:  (1)  the 
relation  between  the  eye  and  the  ground  surface,  (2)  the  information  in  the 
light,  and  (3)  retinal  transformations. 

The  first  phase  requires  a  description  of  the  environmental  event. 

Change  in  forward  speed  expressed  as  a  percent  of  initial  forward  velocity 
describes  relative  displacement  of  the  eye  parallel  to  the  ground  surface. 

This  relative  rate  of  environmental  change  is  specified  as: 

100(x/x)  (expressed  in  %  of  initial  altitude/sec/sec)  (1) 

Change  in  altitude  expressed  as  a  percent  of  initial  altitude  describes  rela¬ 
tive  displacement  of  the  eye  perpendicular  to  the  ground  surface.  This 
environmental  change  is  specified  as: 

100(z/z)  (expressed  in  %  of  initial  altitude/sec)  (2) 

Velocity,  change  in  velocity,  and  rate  of  change  in  velocity  are  not 
visible,  however.  Nor  are  altitude,  change  in  altitude,  and  rate  of  change  in 
altitude.  Differences,  increases,  and  decreases  are  equally  specifiable  under 
conditions  of  no  emitted  or  reflected  illumination,  i.e.,  in  the  dark.  What, 
then,  ifi[  visible? 

The  second  phase  requires  a  description  of  what  takes  place  in  the 
medium  between  the  eye  and  an  illuminated,  textured  ground  surface  during  an 
event.  Four  global  optical  variables  have  been  isolated,  all  capturing 
higher-order  relations  among  relations.  All  have  rates  of  change  identical 
to  the  rate  of  loss  in  speed  or  altitude.  For  the  case  of  movement  along  a 
level  path,  x/z  specifies  flow  rate  and  5c/ z  specifies  flow  deceleration. 

Global  optical  flow  damping  captures  a  higher-order  relation  between  optical 
flow  deceleration  and  optical  flow  rate: 
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observers  x  3  forward  velocities)-  Solid  lines  show  error  rates  for  observers  x  3  forward  velocities).  Solid  lines  show  error  rates  foi 
invariant  values  of  initial  relative  rate  of  optical  change;  broken  invariant  values  of  initial  relative  rate  of  optical  change;  broken 
lines  are  for  different  initial  altitudes.  lines  are  for  different  initial  altitudes. 


100(x/z)/(x/z)  =  100(x/x)  (expressed  in  percent  (3) 
of  initial  forward  velocity/sec) 

Thus,  damping  specifies  a  rate  of  optical  change  that  is  identical  to  the  rate 
of  change  in  forward  displacement  of  the  eye  relative  to  the  ground  surface. 
Percent  decrease  in  forward  velocity  describes  the  relative  change  in  rate  of 
displacement  of  the  eye  at  any  instant.  Percent  decrease  in  optical  flow  rate 
describes  the  optical  rate  of  change  occurring  in  the  ambient  array  along  a 
potential  path  of  observation.  The  rates  of  change  are  identical  because 
optical  flow  rate  change  is  (lawfully,  mathematically)  specific  to  change  in 
rate  of  displacement  of  the  eye. 

For  the  case  of  constant  rate  of  descent,  three  optical  variables  have 
been  isolated:  Global  optical  flow  acceleration  relative  to  initial  optical 
flow  rate  is  specified  as: 

100(Sz/z^)/(§/z)  =  100(z/z)  (expressed  in  %  of  (4) 
initial  eyeheight/sec) 

The  same  holds  for  change  in  global  optical  texture  density  relative  to 
initial  optical  density: 

100(z/g)/(z/g)  =  10Q(z/z)  (expressed  in  %  of  (5) 
initial  eyeheight/sec) 

Warren  (1981b)  has  shown  that  percent  change  in  global  optical  splay  rate 
is  also  identical  to  percent  change  in  altitude  (see  Appendix  A).  Thus  when 
descent  rate  is  constant,  all  three  of  the  optical  variables  isolated  change 
at  the  same  relative  rate  as  that  for  displacement  of  the  eye  relative  to  the 
ground  surface. 

These  relational  identities  also  hold  throughout  the  duration  of  the  event 
even  though  optical  variables  may  change  at  different  rates.  For  example, 
with  a  constant  descent  rate  global  optical  flow  acceleration  accelerates  over 
time,  while  global  optical  texture  density  decreases  at  a  constant  rate. 

Since  the  same  is  true  for  global  optical  splay  angle,  the  relative  rates  of 
change  of  all  three  optical  variables  will  be  the  same  at  reaction  time,  i.e., 
equal  to  the  relative  rate  of  change  in  altitude  at  that  time.  Therefore, 
under  these  conditions  the  three  candidates  for  functional  optical  invariants 
are  indistinguishable  in  terras  of  relative  rate  of  change.  This  is  a  good 
object  lesson  in  the  redundancy  of  optical  information  specifying  any  given 
event . 

The  third  phase  is  a  consideration  of  the  proximal  transformation  as  the 
retina  moves  forward  along  the  path  of  observation  sampling  continuously  the 
change  in  optical  flow.  As  the  eye  loses  velocity,  while  moving  through  the 
stable  optic  array,  the  relative  rate  of  change  of  optical  discontinuities 
focussed  on  the  retina  will  be  identical  to  the  environmental  rate  of  change. 
Change  in  eyeheight  will  produce  a  continuous  proximal  transformation  as  the 
retina  moves  downward  along  the  path  of  observation  sampling  continuously  the 
change  in  optical  flow  rate,  density,  and  splay  angle.  The  rate  of  change  of 
positions  on  the  retina  past  the  optical  discontinuities  focussed  on  the 
retina  will  be  identical  to  the  optical  and  environmental  rates  of  change. 

This  last  step  completes  the  chain  of  specificities  for  stimulation  from 


environment,  through  medium,  to  retinal  variables,  and  allows  perception  to 
be  specific  to  the  relation  between  the  eye  and  the  ground  surface.  The  demon¬ 
stration  of  functional  invariants  shows,  in  turn,  that  the  perceptual  system  is 
sensitive  to  information  specifying  relative  rates  of  displacement. 

The  results  thus  support  the  concept  of  transformational  realism,  i.e., 
when  changes  in  stimulation  appropriately  specify  the  nature  of  an  environ¬ 
mental  event,  there  can  be  a  correspondence  between  perception  and  the  event. 
When  adequate  transformational  information  is  made  available,  the  chain  of 
specificities  will  hold  regardless  of  the  level  of  detail  realism,  as  long  as 
a  minimal  density  of  optical  discontinuities  is  available.  Exactly  what  these 
limits  are,  as  well  as  how  optical  invariants  are  used  in  guiding  locomotion, 
pose  problems  for  future  study. 
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Appendix  A 

The  environmental  and  optical  variables  are  defined  below  following  the 
systematization  by  Warren  (1981a, b).  A  dot  over  a  symbol  indicates  that  the 
variable's  value  changes  as  a  function  of  time. 

Environmental  variables . 

Altitude  =  z  (in  m). 

Forward  velocity  =  x  (in  m/sec). 

Deceleration  rate  ■  x  (in  m/sec/sec). 

Descent  rate  ■  z  (in  m/sec). 
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Path  velocity  =  S  =  Vx  +  z  (in  m/sec). 

Ground  unit  (surface  texture  element)  size  =  g  (in  m). 

Lateral  ground  distance  of  a  surface  edge  from  the  optical  locus 
=  y  (in  m). 

Glide  angle  =  arctan  (z/x)  (in  deg). 


Optical  variables ■ 

Global  optical  flow  rate  for  level  path  =  x/z  (in  eyeheights/sec) . 

Global  optical  flow  deceleration  =  x/z  (in  eyeheights/sec/sec) . 

Global  optical  flow  damping  =  100(x/x)  (in  %  of  an  eyeheight/sec) . 

Global  optical  flow  rate  for  glide  path  =  S/z  (in  eyeheights/sec). 

Global  optical  flow  acceleration  (for  the  special  case  of  constant  path 
speed)  =  100(Sx/z2)  (in  %  eyeheight/sec/sec) . 

Global  optical  texture  density  =  z/g  (in  ground  units/eyeheight). 

Global  optical  splay  rate:  Consider  a  set  of  surface  texture  edges  parallel 
to  each  other  and  parallel  to  the  direction  of  rectilinear  locomotion.  The 
optical  projections  of  the  parallel  edges  intersect  at  a  common  vanishing 
point  on  the  horizon.  The  optical  splay  of  an  edge  line  is  defined  to  be  the 
angular  separation  of  the  optical  projection  of  the  line  from  a  line  perpen¬ 
dicular  to  the  horizon  and  passing  through  the  common  vanishing  point.  The 
latter  line  specifies  the  direction  of  locomotion,  which  may  or  may  not  coin¬ 
cide  with  surface  texture  edges.  In  addition  to  the  presence  of  surface 
texture,  optical  splay  depends  only  on  height  of  the  optical  locus  above  the 
ground  surface  and  lateral  ground  distance  of  the  edge  line  under  considera¬ 
tion  from  the  optical  locus. 

Splay  angle  is  given  by 

SP  =  arctan  (y/z). 

For  vertical  change  in  rectilinear  locomotion  with  no  lateral  velocity,  the 
rate  of  change  of  splay  angle  is  given  by 

SP  =  (z/z)  cos  S?  sin  SP. 

Optical  splay  rate,  then,  depends  only  on  the  relative  rate  of  change  in  alti¬ 
tude  and  the  splay  angle  of  a  particular  projection  line.  It  is  independent 
of  forward  velocity.  For  a  given  change  in  altitude,  the  more  a  projection 
line  is  splayed  from  the  line  specifying  the  direction  of  locomotion,  the 
smaller  the  change  in  splay  angle.  The  relative  rate  of  change  in  altitude 
is  a  scaling  factor  which  applies  equally  to  all  splay  lines  within  an  optic 
array,  and  is  therefore  an  index  of  global  optical  splay  rate: 

GSP  =  z/z  (in  eyeheights/sec) . 
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SCALED  SUBJECTIVE  COMPLEXITY  OF  LOW  LIGHT  LEVEL  TELEVISION  AND  FORWARD  LOOKING 
INFRA  RED  DISPLAYS  APPLIED  TO  COMPUTER  IMAGE  GENERATED  SIMULATION. 
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SCALED  SUBJECTIVE  COMPLEXITY  OF  LOW  LIGHT  LEVEL  TELEVISION  AND  FORWARD  LOOKING 
INFRARED  SENSOR  DISPLAYS  APPLIED  TO  COMPUTER  IMAGE  GENERATED  SIMULATION* 

ABSTRACT 

Nine  subjects  psychos* trlcslly  seeled  photographic  sets  of  Low  Light  Level  Tele¬ 
vision  (L3TV)  and  Forward  Looking  Infrared  (FLIR)  displays  of  the  ease  target-areas 
at  pre-sunset  along  an  equal  interval  continuum  of  scene-complexity.  After  each 
complexity  scaling,  subjects  ware  debriefed  as  to  their  sealing  criteria.  Subjects 
also  matched  both  sets  of  sensor  display  photographs  with  wider-angle,  color  photo¬ 
graphs.  Analyses  Included  significance  tests  of  different  levels  of  scene  coaple- 
xlty,  correlations  between  scales,  and  content  analyses  in  which  scaling  criteria 
were  combined  into  descriptor  sets  frem  which  visual  parameters  and  physical  mea¬ 
sures  amenable  to  CIG  simulation  were  derived.  While  complexity  sealing  of  FLIR 
displays  was  reported  to  be  a  different  and  more  difficult  task,  subjects  matched 
FLIR  displays  with  color  photographs  faster  and  with  fewer  errors.  Findings  support 
a  CIG  simulation  of  L3TV  sensor  displays  using  an  optical  array  of  edges  and  sur¬ 
faces  with  the  degree  of  realism  determined  empirically  using  trade-off  parameters. 

While  moderate  correspondence  was  found  between  L3TV  and  FLIR  display  complexity, 
with  FLIR  displays  subjects  revealed  different  expectancies,  were  more  mission  ori¬ 
ented,  less  certain  of  scaling  criteria,  and  appeared  to  use  target  signatures  am 
cues.  Hence,  realistic  simulation  of  FLIR  displays  using  the  same  CIG  data  base  is 
uncertain  unless  restricted  to  tiae-envelopea  when  the  two  sensors  present  visually 
equivalent  displays. 


INTRODUCTION 

Background .  More  is  demanded  today  of  Electro-Optical  (E/0)  sensor  display  tech¬ 
nology  than  ever  before  including  enhancement  of  damage  assessment  missions,  flight 
control  reconnaissance,  and  navigation  tasks  as  terrain  avoidance/following- (1,2) 

A  dynamic,  functionally  interactive  Sensor  Simulator  to  present  Low  Light  Television 
(L3TV)  and  Forward  Looking  Infrared  (FLIR)  views  during  very  low,  very  fast  flights 
by  means  of  computer  image  generation  (CIG)  is  under  developement  at  the  Air  Force 
Human  Resources  Laboratory  at  WPAFB. 

A  fundamental  issue  in  all  simulator  development  is  determining  what  is  re¬ 
quired  for  positive  transfer-of-training  to  operational  environments  on  a  cost 
effective  basis.  Despite  plans  for  extensive  use  of  simulators  and  ground  trainers, 
there  is  a  serious  lack  of  "hard"  human  performance  data  (3)  while  little  empirical 
data  exists  about  E/O  display  characteristics  and  operator  performance,  especially 
at  altitudes  under  1000  feet  and  speeds  in  excess  of  400  mph. (4)  Although  diverse 
opinions  exist  about  what  E/O  senso.'  simulation  should  include.  Brig.  Gen.B.K. 

Partin  zeroed  in  on  the  fact  that  the  effect  on  operator  performance  of  varying 
degrees  of  realism  must  be  established.  (5)  Thus  an  assessment  of  human  responses 
to  L3TV  and  FLIR  scenes  is  a  purpose  of  this  research.  Since  Rosell  and  Willson 
cited  several  display  degrading  factors  beyond  the  control  of  sensor  designers  and, 
possibly,  beyond  deciphering  by  an  operator  (4) ,  a  second  purpose  was  to  establish 
a  method  to  identify  visual  parameters  which  should  be  included  in  a  FLIR/L3TV 
Sensor  Simulator. 

Bunker  and  Heeschen  asserted  that  with  computer  generated  imagery, realism  is  a 
function  of  the  complexity  of  the  scene  that  can  be  shown. (6)  CIG  techniques  repre¬ 
sent  real  world  features  by  storing  3D  information  of  edges  and  surfaces  to  produce 
a  dynamic  display  on  a  2D  screen  by  transformation  of  the  stored  data.  Although  the 
state-of-CIG-art  includes  over  5000  edges,  the  level  of  instrumentation  available  at 
the  ASM  Branch  of  AFHRL  permitted  2000  visible  edges.  This  is  the  same  as  in  the 
Advanced  Simulator  for  Pilot  Training  with  which  operators  were  reported  to  experi- 
ence  difficulty  detecting  heights  at  low  altitudes. (4)  To  improve  the  perception  of 
distance  in  simulated  displays  of  open  space,  Gibson  suggested  the  use  of  texture 
gradients. (7)  The  application  of  texturing  in  simulated  L3TV/FLIR  displays  would 

1  Research  was  Initiated  while  the  auther  waa  an  ASEI-USAF  Swaaer  Research  Fails* 
at  AFHRL,  WI»FB,  Oh.  and  finished  at  Florida  Intsmatlenal  University,  Klaal,  FI. 
under  alnigrant  #77-3242  free  The  Air  Faroe  Offioe  of  Scientific  Research,  AFSD, 

Belling  AFB,  D.C. 
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increase  the  required  number  of  CIG  edges  and  surfaces  as  would  other  features  as¬ 
sociated  frequently  with  complexity. (8) 

Because  programming  time  and  hardware  costs  increase  considerably  as  the  number 
of  scene  edges  increase,  scene  complexity,  held  to  be  a  major  determiner  of  realism, 
is  also  a  major  determiner  of  costs.  Hence,  trade-offs  must  be  sought  which  coincide 
with  visual  parameters  least  important  for  positive  transfer-of-training.  CIG  para¬ 
meters  available  for  trade-off  consideration  include  edge  smoothing,  contouring, 
curvature,  the  addition  of  noise,  number  of  grey  levels,  foliage  masking,  and 
weather  algorithms. 

Objectives.  If  differences  in  scene -complexity  can  be  established,  computer  hard¬ 
ware  and  programming  trade-offs  could  be  determined  empirically  by  simulating  a 
scene  of  known  subjective  complexity,  varying  CIG  parameters,  and  measuring  perfor¬ 
mance.  Thus,  specific  research  objectives  were: 

-to  scale  the  complexity  of  still  L3TV  and  FUR  scenes. 

-to  identify  scenes  at  significantly  different  levels  of  complexity. 

-to  determine  major  perceptual  factors  associated  with  complexity  of  L3TV 
FLIR  displays. 

-to  relate  perceptual  with  physical  factors  amenable  to  CIG  simulation. 
Assumptions.  Subjects  are  assumed  to  be  replicas  of  each  other  and  able  to  produce 
unidimentional  scene -complexity  scales  with  the  variability  in  their  judgements  dis¬ 
tributed  randomly.  For  a  given  stimulus  scene  the  best  value  of  scene-complexity  is 
mean  scale  value,  thus  significant  differences  can  be  computed  (9)  and  a  scale  of 
scene-conqplexity  developed  for  a  set  of  L3TV  or  FLIR  displays  producing  a  monotonic 
relationship  with  physical  dimensions  of  the  stimulus  materials. (10) 

THE  METHOD 

Stimulus  Materials.  The  E/O  Sensor  Lab  of  the  Air  Force  Avionics  Laboratory  began 
recording  on  to  video  tape  FLIR  and  L3TV  sensor  displays  of  16  local  target-areas 
taken  from  atop  the  AFAL  Tower  at  Wright  Patterson  Air  Force  Base  in  late  winter, 
1976.  Although  the  FLIR  system  was  a  prototype  model,  the  L3TV  system  was  in  stan¬ 
dard  production  and  use.  Stimulus  materials  for  the  present  research  consisted  of 
two  sets  of  sixteen  8"xl0"  glossy  photographs  of  still  video  displays  of  16  tar¬ 
get-areas  recorded  on  5/11/76  during  one  video  taping  between  1915  and  1922  o’’clock 
in  the  time  envelope  classified  as  Pre-Sunset.  One  set  consisted  of  L3TV  displays, 
the  other  FLIR  displays.  The  photographs  of  still  video  tapes,  displayed  on  a  CONRAC 
QQA  21”  monitor,  were  made  with  a  Hasselblad  camera  and  tripod.  Atmospheric  and 
weather  conditions  at  the  time  of  the  taping  were:  Temperature-65. 6°  F. ;  Relative 
Humidity-32%;  Visibility-10  miles, -Cloud  Cover-Clear;  Type  of  day-Hazy. 

Both  L3TV  and  FLIR  sensor  systems  can  present  either  a  wide  or  narrow  field  of 
view,  dimensions  depending  upon  the  particular  system,  model,  and  operating  condi¬ 
tions.  Since  the  sensor  systems  at  the  AFAL  Tower  present  a  narrow  field,  the  stim¬ 
ulus  photographs  captured  views  of  the  environment  subtending  6  20"  -  7^40'  of 
horizontal  arc  and  4°40'  -  5°40'  of  vertical  arc  at  the  lens. 

At  some  time-envelopes  during  daylight  hours  under  good  weather  conditions, 

L3TV  and  FLIR  sensors  can  present  displays  which  are  nearly  equivalent, visually. 

Both  displays  can  also  look  much  like  the  normal,  real  world  visual  scene  and,  at 
times,  present  even  an  enhanced  view.  At  other  time-envelopes,  however,  L3TV  and 
particularly  FLIR  sensors  yield  displays  with  predictably  different  characteristics, 
unlike  daylight  visual  scenes.  For  example,  FLIR  sensors  are  noticfeably  responsive  to 
relative  temperature  changes  which  occur  around  sunset  as  the  heating  property  of  the 
sun  is  withdrawn.  A  period  of  thermal  transition  is  regularly  displayed.  This  tran¬ 
sition  period  could  be  fairly  brief,  depending  upon  climate  and  other  factors,  but 
it  is  not  necessarily  nor  is  it  of  constant  duration.  Some  environmental  features 
which  have  been  reflecting  heat  from  the  sun  cool  down  and  their  natural  pattern  of 
heat  emissivity  begins  to  register.  Among  environmental  features  which  generate 
no  heat  of  their  own,  some  cool  quickly  while  others  dissipate  heat  stored  during 
the  day  quite  slowly  over  a  period  of  hours.  Similarly,  in  a  night  environment 
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standard  L3TV  sensors  can  be  over-sensitive  to  small  light  sources.  The  phenomenon 
of  "blooming",  in  which  a  light  may  become  considerably  enlarged,  may  occur.  If 
corrections  are  not  made,  a  light  can  bloom  to  flood  the  entire  display. 

Since  the  displays  used  in  this  research  were  recorded  during  the  time-enve¬ 
lope  of  Pre-Sunset  in  late  spring,  of  the  two  sensors  the  L3TV  display  most  close¬ 
ly  approximated  a  pre-dusk  visual  scene  with  some  typical  reel  world  optical  features 
such  as  elongated  shadows  and  rounded,  opaque  foliage;  no  lights  were  blooming.  The 
FLIP  display  revealed  the  beginning  of  a  thermal  transition  period.  Concrete  objects 
appeared  slightly  warmer  than  the  earth,  while  close  foreground  trees  began  to  show 
a  natural  heat  emission  pattern  with  lightish  trunks  and  branches.  Some  heat  emit¬ 
ters,  undetectable  during  the  day,  appeared  as  whitish  blobs  or  smears  when  seen 
through  foliage.  At  times,  the  level  of  water  was  faintly  discernable  within  a  near¬ 
by  water  tower. 

The  original  stimulus  materials  are  classified  and  cannot  be  presented.  To  give 
a  rough  impression  of  the  content  of  the  FLIR  and  L3TV  displays.  Figure  1  contains 
reproductions  of  daylight  color  photographs  of  each  target-area,  numbered  and  titled 
as  during  taping.  While  each  picture  has  been  cropped  to  the  approximate  narrow 
field  of  view,  photographs  of  FLIR  and  L3TV  sensor  displays  are  not  visual  equiva¬ 
lents  of  these  surrogates. 

Procedure:  Experiment  1.  In  seperate  sessions  nine  subjects  scaled  the  two  sets  of 
photographs  along  a  metathetic  continuum  of  scene  complexity  divided  into  7  equal 
intervals. (11)  The  subject  sat  before  a  table  across  which  seven  equally  spaced, 
numbered  categories  were  indicated  with  #1  labeled  "Least  Complex,"  #4  "Average 
Complexity,"  and  #7  "Most  Complex."  The  subject  was  informed  that  the  stack  of 
photographs  immediately  in  front  of  him  were  taken  from  L3TV  (or  later, FLIR)  video 
tapes  of  16  local  target-areas  recorded  at  Pre-Sunset  in  late  spring.  He  was  asked 
to  sort  the  photographs  into  one  of  the  7  categories  of  complexity,  making  the 
steps  between  categories  as  equal  as  possible  with  at  least  1  photograph  assigned  to 
each  category.  Subjects  could  perform  the  scaling  task  in  any  manner  they  chose  and 
take  as  long  as  they  wished  until  they  were  satisfied  with  their  scaling.  All  sub¬ 
jects  scaled  photographs  of  L3TV  sensor  displays  first  and  were  then  debriefed.  Lat¬ 
er,  the  process  was  repeated  with  photographs  of  FLIR  sensor  displays. 

Debriefings .  After  a  subject  indicated  he  was  satisfied  with  his  scaling,  photographs 
in  all  categories  were  spread  out  vertically  for  maximum  visibility.  Comments  were 
elicited  from  the  subject  by  asking  a  series  of  questions  such  as:  "What  were  vou 
thinking  about  as  you  arranged  the  pictures  in  these  categories?  How  did  you  decide 
which  picture  should  go  where?  What  made  you  decide  this  picture, say,  belonged  here 
instead  of  there  (pointing)?  Can  you  describe  the  difference  you  found  between 
pictures  you  placed  in  lower  categories  with  those  in  the  middle  or  higher?  "  since 
a  pause  was  taken  between  each  question,  subjects  usually  started  reporting  about 
their  scaling  criteria  by  the  second  question.  Observations  and  a  verbatim  tran¬ 
script  of  comments  made  during  scaling  and  debriefing  were  recorded. 

Procedure:  Experiment  2.  Subjects  matched  both  sets  of  FLIR  and  L3TV  display  photo¬ 
graphs  previously  scaled  for  complexity  with  sixteen  3"x5y  color  photographs  con¬ 
taining  within  them  target-areas.  Subjects , handed  the  stack  of  small  color  photo¬ 
graphs,  confronted  a  covered  table  on  which  16  FLIR  (or  L3TV)  photographs  were 
arranged  in  a  4x4  display.  The  arrangements  of  FLIR  and  L3TV  photographs  and  the 
sequence  of  color  photographs  were  randomized  for  all  presentations.  A  subject's 
task  was  to  match  the  sensor  display  photos  as  quickly  as  possible  by  placing  a 
small  color  photo  containing  the  scene  on  top  of  each.  At  a  given  signal,  the  cover 
was  removed  and  a  stop  watch  started  which  was  stopped  when  the  subject  finished. 
While  the  subject  stepped  out  of  the  room,  the  Experimenter  noted  any  errors,  set 
up  the  second  arrangement,  covered  it,  and  rearranged  the  color  photographs.  Match¬ 
ing  order  was  conterbalanced  among  subjects. 

Subjects.  Time  and  cost  precluded  subject  groups  of  users  and  trainees  reported  to 
be  the  best  judges  of  training  requirements. (12,13)  The  group  of  9  subjects  was  a 
sample  of  convenience  selected  on  the  basis  of  availability  combined  with  knowledge 
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and  experience  with  E/0  sensors.  It  included  2  Human  Factors  Psychologists,  4 
Computer  Engineers,  and  3  E/0  Sensor  Experts.  Although  visually  representative  of 
user  groups,  the  subjects'  representativeness  in  terms  of  expectancies,  cognitive 
associations,  and  use  of  perceptual  cues  is  indeterminant. 

Content  Analyses.  To  determine  major  factors  associated  with  subjective  complexity 
of  L3TV  and  FLIR  displays,  transcripts  of  each  subject's  comments  during  scaling 
and  debriefing  were  subjected  to  a  content  analysis.  Primary  scaling  criteria  were 
defined  as  descriptors  or  key  words  used  by  a  subject  to  describe  the  overall  con¬ 
tinuum,  or  to  discuss  the  major  differences  between  end  points  of  the  scale.  Second¬ 
ary  criteria  were  defined  as  descriptors  mentioned  secondarily,  or  used  to  distin¬ 
guish  between  intermediate  scaling  categories.  Concepts  used  as  criteria  in  describ¬ 
ing  the  scaling  task  by  a  majority  of  subjects  were  included  in  the  analysis.  The  two 
content  analyses  consisted  of  the  following  general  steps: 

1)  Examination  of  the  verbatim  comments  and  debriefing  responses  with  preliminary 
classification  into  primary  and  secondary  descriptors  stressed  by  each  subject. 

2)  Enumeration  of  primary  and  secondary  scaling  criteria  adduced  by  combining 
similar  descriptors  from  all  subjects. 

3)  An  attempt  to  synthesize  content  analysis  descriptors  into  physical  para¬ 
meters  and  derive  physical  measures  useful  in  CIG  applications. 

RESULTS  AND  DISCUSSION 

L3TV  Complexity  Scaling.  Frequency  distribution^  of  the  subjects'  categorizations  of 
each  L3TV  scene  with  the  mean  complexity  value, X,  and  standard  deviation, s,  are 
presented  in  Table  1.  The  scenes  are  ordered  according  to  their  mean  complexity  from 
low  scale  values  indicating  less  complexity  to  greater  complexity  represented  by 
higher  scale  values.  The  non-overlapping  distributions  enclosed  in  boxes  identify 
scenes  satisfying  the  Guttman-like  steps  required  in  Scalogram  Analysis  of  L3TV 


Table  1.  Ordered  Frequency  Distributions  of  L3TV  Complexity 


Scenes 

Scale 

Values 

# 

Abbrievated  Title 

A 

Ji 

7 

Housing  Area  (landscape) 

i.i 

0.3 

16 

Trebein  Site  (landscape) 

1.7 

0.9 

8 

Apartment  Complex,  Trees 

1.9 

0.8 

12 

Church  in  Woods 

2.2 

0.8 

6 

Microwave  &  Water  Towers 

2.3 

1.2 

15 

1-675  Overpass 

3.0 

0.7 

9 

W.S.U.  Water  Tower 

3.4 

0.7 

11 

W.S.U.  Microwave  Dish 

3.4 

0.7 

13 

D.P.&L.  Sub-station 

3.8 

1.3 

14 

Active  Cement  Plant 

4.1 

0.8 

10 

Fairborn  Water  Tower 

4.4 

0.9 

4 

Fairborn  Grain  Elevator 

5.4 

0.5 

3 

Inactive  Cement  Plant 

5.4 

1.0 

2 

Steam  Plant 

5.6 

0.9 

1 

Patterson  Field  Hangars 

6.6 

0.5 

5 

Base  Hospital 

6.9 

0.3 

Complexity  Categories 
least  most 

1  2  3  4  5  6  7 


display  complexity. (14)  Figure  2  graphically  presents  scaled  L3TV  displays  and  the 
results  of  two-tailed  t-tests  of  differences  between  the  means. (15,16)  Four  signi¬ 
ficantly  different  levels  of  L3TV  display  complexity  at  the  99%  level  of  confidence 
(p<. 01)  are  identified  by  scene  numbers  enclosed  in  solid  lines.  Six  significantly 
different  levels  of  L3TV  display  complexity  at  the  95%  level  of  confidence  (p<.05) 
are  established  by  the  inclusion  of  3  additional  scenes  enclosed  in  dashed  lines. 

To  permit  a  rough  appraisal  of  changes  in  scene  content- areas  across  six  levels  of 
complexity,  scaled  representative  photographs  of  the  target-areas  at  the  6  levels 
identified  in  Figure  2  are  presented  in  Figure  3.  Despite  the  loss  of  resolution  in 
the  surrogate  photographs,  the  pre-sunset  L3TV  sensor  displays  also  had  diminished 
resolution  but  with  different  shadows,  figure-ground  relationships,  less  freedom 
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V 


Levels  of 
Complexit 

#7 


|  #12  j 

l - 1 


#15 


Scale 

Least  Complex 

1.0-, 

1.2- 

1.4- 

1.6- _ 

1.8- _ 

2.0- 

- 2.2-- 

2.4- 
2.6- 
2.8- 

- 3.0 - 

3.2- 

3.4 - 

3.6- 

3.8 - 

4.0- 

4.2- 

4.6- 

4.8- 
5.0- 

5.2- 

- 5.4 - 

- 5.6 - 

5.8- 
6.0- 

6.2- 

6.4- 

- 6.6 - 

6.8- 
7  ,0J 

Most  Complex 


Scenes  (Mean  Scale  Value) 


Title 

Value 

-  7. 

Housing  Area 

1.1 

-16. 

Trebein  Site 

1.7 

-  8. 

Apartment  Complex 

1.9 

H: 

Church  in  Woods  __ 
Microwave  &  water  Twrs 

.hi 

-15. 

1-675  Overpass 

3.0 

•  9. 

W.s.U.  Water  Tower 

3.4 

11. 

WS.U.  Microwave  Dish 

3.4 

-13. 

D.P.SL.  Sub-station 

3.8 

-14. 

Active  Cement  Plant 

4.1 

-10. 

Fairborn  Water  Tower 

4.4 

4.  Fairborn  Grain  Elevator  5.4 
3.  Inactive  Cement  Plant  5.4 
2.  Steam  Plant  5.6 


1.  Patterson  Field  Hangars  6.6 
5.  Base  Hospital  6.9 


Figure  2. 


Scene  Complexity  Scaling  of  L3TV  Scenes  With  Significantly 
Different  Levels  of  Complexity  Identified 


from  noise,  and  some  edge  distortion. 

L3TV  Content  Analysis  and  Conversion  to  Derived  Measures.  Although  few  subjects  made 
comments  while  scaling  L3TV  displays,  descriptors  elicited  during  debriefings  were 
enumerated  from  transcripts.  Table  2  presents  similar  descriptors  combined  into 
seven  sets  with  the  number  of  subjects  using  the  descriptors  of  each  set  as  primary 
or  secondary  scaling  criteria,  their  combined  total,  and  the  percent  of  the  subject 
group . 

While  one  subject  thought  he  might  have  gotten  a  different  ordering  if  asked  to 
scale  on  "clarity",  the  assumed  unidimensionality  of  L3TV  scene -complexity  appeared 
to  be  dubious  for  only  one  subject  who  discovered  that  he  must  have  used  some  con¬ 
flicting  criteria  which  might  not  be  on  the  same  scale.  Most  subjects  immediately 
focused  on  specific  features  emerging  from  the  figure-ground  relationship  with  100% 
discussing  "objects." 

An  unmet  expectation  derived  from  the  literature  was  that  brightness,  the  "lumi¬ 
nous  output  pattern,"  and  contrast  would  be  mentioned  frequently. (17)  Instead,  only 
one  subject  spoke  of  "broken  light  uniformity."  However,  the  light  pattern  may  have 
been  related  to  the  way  in  which  the  background  was  regarded.  To  the  7  subjects  who 
mentioned  it,  the  background  (terrain, landscape,  foliage)  appeared  relatively  unim¬ 
portant.  Only  one  subject  implied  a  possible  masking  function.  Unbroken  lightish 
areas  were  merely  trees  to  one  subject  while  two  described  the  vegetation  background 
as  non-informational  and  pretty  much  the  same  across  all  scenes.  Three  subjects 
clearly  saw  scenes  comprised  mostly  of  terrain  or  landscape  as  least  complex  while 
another  referred  to  the  "garbage  of  these  surrounds."  Two  subjects  mentioned  "clutter", 
one  referring  to  targets  lost  in  the  "clutter"  of  less  conplex  scenes,  while  the 
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With  iyntheilsed  Paraeetere  and  Derived  Hassuraa. 


HtHAHtreitj  nggmg  hkaswus _ 

I.  The  number  of  man-made  object*  or  object-areas  In  a 

scene. 

(1)  The  nuaber  of  separate,  single  enti¬ 
tles  Ignoring  substructural  aspects, 
attachments,  or  features. 

(2)  The  numbs r  of  single  entitles  Inclu¬ 
ding  substructural  aspects  provided 
the  smaller  feature  can  be  recognised 
as  a  completed  feature  of  the  larger. 

II.  a.  The  ratio  of  the  area  of  man-made  objects  to 
the  combined  areas  of  fol lage/terrain  a  sky.  or 

b.  The  percentages  of  total  scene  area  composed  of 
man-made  objects/  foliage  ft  terrain/  sky. 

(3)  Area  of  man-made  objects. 

14)  Area  of  follage/terrela. 

1 5)  Area  of  sky. 

III.  The  number  of  straight  and  curved  lines  which  make 
up  visible  Internal  and  external  portion*  of  man¬ 
ned*  objects. 

(6)  The  nuaber  of  straight,  curved,  and 
total  external  line*. 

(7)  The  nuaber  of  straight,  curved,  and 
total  Internal  lines. 

IV.  Seen*  quality  as  reflected  In  resolvable  detail. 

(8)  a.  The  shorter  dimension  of  the  smal¬ 
lest  object  in  a  scene  during  (2)  0£ 

b.  The  length  of  the  shortest  line  In¬ 
cluded  In  (6)  or  (7)  above. 


other  described  a  profusion  of  objects  in  the  roost  complex  as  "clutter." 

The  author's  privately  held  notions  that  subjects  would  verbally  relate  inform¬ 
ational  content  with  scene  complexity,  and  that  most  subjects  would  show  a  mission  or 
task  orientation,  received  little  support .  "Information"  was  used  by  3  subjects,  but 
in  different  contexts.  Althouqh  all  subjects  were  or  had  been  associated  in  some  way 
with  the  developemont  of  a  Sensor  Simulator,  during  1.3TV  debrief inqs  only  a  total  of 
4  subjects  mentioned  any  mission-oriented  or  task-performance  words  as  "Tarqet," 
"Recoqnition, "  ”  Identification,"  or  "Detection." 

Before  attendinq  to  physical  parameters  and  measures  evolved  from  the  content 
analysis  in  Table  2,  a  major  limitation  of  the  present  study  must  be  clearly  stated: 
this  research,  restricted  by  the  nature  of  the  particular  stimulus  sets  and  subject 
group  used,  is  best  reqarded  as  a  heuristic,  feasibility  study.  For  example,  in  the 
entire  stimulus  set  there  were  no  natural  geographical  features  or  foliated  objects 
which  stood  out  as  "figures."  In  addition,  with  the  exception  of  patches  of  an  in¬ 
terstate  highway  or  part  of  a  runway,  there  were  few  visible  man-made  alterations  of 
terrain  such  as  cultivated  fields,  orchards,  local  roads,  or  even  fences.  To  satisfy 
simulator  requirements  for  training  in  a  variety  of  flight  missions  under  varying 
circumstances,  a  larger  data  base  would  be  required  for  definite  conversions. 

As  illustrated  in  Table  2,  the  first  physical  parameter  is  based  on  Set  1  con¬ 
taining  responses  from  100%  of  the  subjects.  This  parameter  involves  merely  a  count¬ 
ing  process  following  an  explicit  definition  of  what  constitutes  a  man-made  object. 
Gestalt  principles  on  the  properties  of  figures  plus  grouping  factors  for  partially 
concealed  patterns  should  suffice. (19)  Fach  separate  figure,  no  matter  the  size  or 
partial  obscuration,  could  be  counted  as  one. (20)  Because  figures  are  not  necessar¬ 
ily  the  same  as  "man-made  objects"  (21),  measures  (1)  and  (2)  were  derived  . 


409 


Set  4  (area  of  man-made  objects)  is  related  to  Set  2  (terrain,  foliage, etc. ) 
but  not  necessarily  inversely  because  the  amount  of  sky  visible  varied  with  the  scene. 
Since  the  sky  was  never  cited  as  a  scaling  determiner,  subjects  probably  lumped  sky 
area  in  with  foliage/terrain  as  background  in  this  stimulus  set.  Yet  it  is  also 
probable  that  in  a  terrain  avoidance  task  the  amount  of  visible  sky  would  achieve 
greater  significance.  Sets  2  and  4,  combining  the  responses  of  89%  of  the  subjects, 
were  synthesized  into  Parameter  II.  Two  aspects  of  this  parameter  are  suggested  with 
the  second  expected  to  be  more  applicable  with  a  greater  data  base.  The  three  mea¬ 
sures,  (3),  (4) ,  and  (5),  necessary  to  compute  Parameter  II  could  be  achieved  by 
placing  an  appropriately  fine  grid  over  a  display  photograph  and  counting  area  units. 

Set  3  combines  concepts  of  shape  or  outline  complexity  while  Set  5  is  focused 
on  interior  structures  or  patterns.  Since  both  sets  deal  with  aspects  of  object 
detail,  they  are  related  to  Set  7  (number, kind, and  direction  of  lines)  not  only  be¬ 
cause  visual  detail  is  graphically  displayed  by  the  presence  of  detectable  lines  or 
quick  gradient  changes  but  also  because  edge,  slit, and  line  detectors  are  held  to 
monitor  specific  regions  of  the  retina. (22)  Hence  Parameter  III  incorporates  the 
responses  of  100%  of  the  subjects  and  can  be  enumerated  directly  in  the  two  measures 
(6)  and  (7),  most  meaningful  to  CIG  programmers. 

The  common  thread  running  through  the  descriptors  in  Set  6,  incorporating  res¬ 
ponses  of  56%  of  the  subjects  ,  is  an  apparent  reference  to  image  or  scene  quality. 
Subjectively,  image  quality  of  real-life  scenes  is  influenced  by  many  factors  such 
as  scene  content,  overall  brightness,  contrast  )evels,  texture  gradients,  assigned 
task,  noise, etc.  A  similarly  extensive  list  of  physical  parameters  has  been  identi¬ 
fied  as  determiners.  (10)  Yet  the  traditional  psychophysical  approach  tends  to  put 
first  emphasis  on  resolution,  as  do  many  current  optical,  photographic,  and  electron¬ 
ic  display  approaches. (23,24)  The  human  ability  to  resolve  video  targets  appears  to 
vary  as  a  function  of  the  geometry  of  the  test  pattern  while  varying  system  resolu¬ 
tion  appears  to  be  a  function  of  the  number  and  length  of  lines  and  geometries  of 
test  charts. (10)  Thus,  attempting  the  simplest  first.  Parameter  IV  relates  scene 
quality  with  resolvable  detail.  Since  the  visual  acuity  of  the  subjects  exceeded 
system  resolution  revealed  in  the  set  of  stimulus  photographs ,  a  logical  initial  ap¬ 
proach  to  scene  or  image  quality  would  be  to  measure  the  smallest  extent  of  line  or 
objects  in  each  scenewwhich  can  be  recognized  as  such  in  measures  (2) ,  (£) ,  and  (7) . 
Toward  an  L3TV/CIG  Property  List. Rather  than  taking  the  physical  parameters  syn¬ 
thesized  from  the  content  analysis  and  their  constituent  measures  presented  in 
Table  2  at  face  value,  each  measure  should  be  validated  by  pairing  the  value  found 
for  each  scene  on  the  8  derived  measures  with  that  scene’s  mean  L3TV  scene  complex¬ 
ity  score  and  applying  any  one  of  several  correlational  techniques. (25,26)  In  gen¬ 
eral,  only  measures  found  to  have  moderate  to  high  positive  correlation  coefficients 
or  good  predictive  ability  should  be  retained.  By  such  a  culling  an  initial  L3TV/ 

CIG  "Property  List"  could  be  established  indicating  which  CIG  parameters  are  most 
important  for  simulation  and  hence,  must  be  included  in  a  Sensor  Simulator  to  max¬ 
imize  transfer-of-training. 

Scaling  FLIR  Complexity.  Two  outstanding  features  distinguished  the  FLIR  experimental 
sessions  from  L3TV  scaling.  The  first  was  initial  subject  response  to  the  scaling 
task;  many  started  talking  early  in  the  process  of  scaling.  The  following  excerpts 
from  comments  of  six  subjects  demonstrate  that  despite  their  knowledge  and  familiar¬ 
ity  with  FLIR  sensor  displays,  a  majority  seemed  ill  prepared:  "Things  are  quite  dif¬ 
ferent.  .  .considerably  harder!"  "Holy  Cow!  This  is  a  different  ball  game."  "This  is  ' 
much  harder... not  the  same  job!"  "This  brightness  is  too  high!"  "..more  difficult... 
than  earlier. . (some)  overexposed."  "..just  isn't  the  same  task." 

The  second  feature,  discovered  at  the  end  of  a  debriefing,  was  that  the  one  sub¬ 
ject  who  had  been  a  pilot  scaled  the  FLIR  photographs  as  if  he  were  on  the  "tactical 
mission  of  reconnaissance  strike  utility."  Since  the  subject  volunteered  to  rescale 
the  set,  it  was  so  arranged.  Subsequent  debriefing  revealed  that  in  his  second  scal¬ 
ing  the  subject  had  assumed  "the  strategic  mission  of  low  level  ingress."  While  this 
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subject  had  displayed  a  slight  mission  orientation  in  L3TV  scaling,  scaling  FLIR 
scene- complexity  became  inextricably  related  to  specific  mission  performance. 

Because  none  of  the  other  subjects  had  demonstrated  similar  behavior,  whether  to  in¬ 
clude  or  exclude  this  subject's  data  in  the  scaling  analysis  became  an  issue.  Examin¬ 
ation  of  the  subject's  initial  FLIR  scaling  (assumed  tactical  mission)  revealed  dis¬ 
parate  categorization  relative  to  the  response  distribution  of  the  other  8  subjects 
on  56%  of  the  scenes.  Examination  of  his  second  scaling  (assumed  strategic  mission) 
revealed  an  even  greater  disparity  on  69%  of  the  scenes.  Inclusion  of  either  scal¬ 
ing  affected  some  means  and  standard  deviations  markedly.  Hence,  this  subject  was 
declared  a  non-replicate  and  his  data  excluded  from  the  scaling  analysis. 

Ordered  frequency  distributions  of  the  remaining  8  subjects'  complexity  scalings 
for  16  FLIR  sensor  display  photographs  are  presented  in  Table  3  with  their  mean  com¬ 
plexity  scale  value,  X,  and  standard  deviation, s.  As  in  earlier  analysis,  non-over- 
lapping  distributions  encased  in  boxes  reveal  3  Guttman  steps  of  scene  complexity. 
"However,  7  FLIR  scenes  are  encompassed  in  these  steps  in  contrast  to  only  4  L3TV 
scenes  earlier.  Another  comparison  with  Table  1  reveals  that  the  frequency  distri¬ 
bution  for  FLIR  photographs,  although  based  on  one  less  subject,  are  a  bit  more 
spread-out  than  those  for  L3TV,  particularly  as  scene  complexity  increases. 


Table  3.  Ordered  Frequency  Distributions  of  FLIR  Complexity.  N-8 
Scenes _  Scale  Values  Complexity  Categories 
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The  excluded  ninth  subject's  responses  are  illustrated  in  Table  3  by  supranu- 
merals.  The  superscript  1  indicates  the  subject's  response  assuming  "the  tactical 
mission  of  reconnaissance  strike  utility,"  the  superscript  2  that  for  "the  strategic 
mission  of  low  level  ingress."  Had  the  ninth  subject's  first  scaling  been  included, 
two  Guttman  steps  would  have  resulted;  had  his  second  been  included,  none. 

As  in  the  earlier  treatment  of  L3TV  scenes,  two-tailed  t-tests  of  the  difference 
between  means  were  performed  revealing  four  significantly  different  levels  of  FLIR 
scene  complexity  at  the  99%  level  of  confidence.  These  are  illustrated  in  Figure  4 
by  scenes  whose  numbers  are  enclosed  in  solid  lines.  The  inclusion  of  FLIR  scenes 
differing  in  complexity  at  the  95%  level  of  confidence  (p  .05)  did  not  increase  the 
numbers  of  complexity  levels:  it  merely  provided  a  few  alternative  scenes,  enclosed 
in  dashed  lines,  at  the  four  levels  of  FLIR  scene  complexity  already  established. (16) 
Although  FLIR  sensor  displays  are  not  their  visual  equivalent,  surrogate  day¬ 
light  photographs  illustrate  4  levels  of  FLIR  scene -complexity  in  Figure  5. 

FLIR  Content  Analysis.  Most  subjects  found  complexity  scaling  of  FLIR  displays  not 
only  more  difficult  than  L3TV  scaling,  but  also  a  different  kind  of  task.  Although 
two  subjects  reported  that  they  used  the  same  criteria  as  earlier,  one  expressed  dis¬ 
satisfaction  with  his  scaling.  Others  appeared  less  certain  than  before  about  what 
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Most  Complex 

Figure  4.  Complexity  Scaling  of  FLIR  Scenes  Based  on  N  =  8 

With  Significmtly  Different  Levels  of  Complexity. 

criteria  they  had  actually  used.  Two  reported  that  criteria  appeared  to  change  as 
they  moved  up  the  scale;  another  reported  using  conflicting  criteria.  Due  to  the 
above  and  to  evidence  of  increased  mission  orientation,  the  ninth  subject's  comments 
during  his  first  FLIR  debriefing  were  included  in  the  content  analysis. 

Perhaps  because  subjects  sometimes  had  difficulty  determining  where  the  back¬ 
ground  left  off  and  objects  began,  the  background  in  FLIR  displays  was  not  treated  as 
uninportant.  Several  subjects  mentioned  masking  or  smearing  of  objects  into  the 
background.  Others  chose  to  treat  whited-out  portions  of  buildings  in  the  immediate 
foreground  as  ground.  One  subject  specifically  noted  the  area  taken  up  by  the  Sky. 

The  assumed  unidimensionality  of  a  scehe-complexity  continuum  appears  question¬ 
able  with  FLIR  displays  because  many  subjects  showed  increased  uncertainty  about, 
their  scaling  criteria.  Several  spoke  of  flipping  over  or  folding  over  the  scale  or 
parts  of  the  scale.  While  a  mission  orientation  was  fully  manifested  by  the  ninth 
subject,  a  similar  association  with  FLIR  displays  appears  to  influence  other  subjects 
because  the  use  of  mission-oriented  and  task-performance  words  occurred  with  greater 
frequency  than  in  L3TV  debriefings.  For  example,  "Target"  was  used  by  twice  as  many 
subjects  as  before.  "Recognition,"  previously  cited  by  one,  was  referred  to  by  five. 
"Identification"  was  mentioned  by  only  one  of  the  two  using  it  earlier,  but  here  with 
reference  to  "check-points"  and  "centers  of  man-made  somethings."  "Detection"  was 
not  used  by  any  subject  after  scaling  FLIR  photographs - but  neither  were  there  ref¬ 

erences  to  sub-objects,  sub- structures,  or  sub-patterns.  "Information"  was  used  more 
frequently  with  FLIR  than  with  L3TV  displays.  Two  subject’  used  it  synonymously  with 
object  detail.  Three  referred  to  more  global  concepts  of  display/terrain/or  cultural 
information. 

The  luminours  output  pattern  or  scenic  pattern  of  contrast  levels  (17)  ,  while 
not  cited  directly,  appeared  to  give  many  subjects  difficulty.  Although  intellec¬ 
tually  prepared  to  scale  the  heat  patterns  of  FLIR  displays,  several  subjects  failed 
initially  to  recognize  the  whiteness/brightness  of  some  display  areas  as  information 
cues  of  relative  heat  or  fuzzy  edges  of  of jects  as  heat  emissivity.  A  couple  of 
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Figure  $•  Surrogate  Daylight  Riotographs  of  FLIH  Target-Areas  at  Four  Signifi 
cantly  Different  Levels  of  Scene  Coaplexity.  (0*8) 


subjects  had  to  remind  themselves  what  light  areas  meant.  Some  subjects  complained 
about  the  washing-out,  smearing,  or  obliteration  of  figural  detail  by  brightness  or 
being  "over-exposed."  In  short,  infrared  information  was  often  treated  as  noise. 
Despite  prior  experience,  the  predominant  "gut  level"  response  of  the  subject  group 
was  as  if  they  had  expected  a  display  of  a  normal  visual  scene  where  brightness  meant 
light  and  not  heat. 

Following  the  same  procedure  as  earlier,  a  content  analysis  was  performed  on  the 
transcripted  comments  and  the  results  are  presented  in  Table  4.  Descriptors  in  pa¬ 
rentheses  are  those  of  the  ninth  subject.  With  the  exception  of  Set  8,  descriptor 
sets  are  presented  in  Table  4  which  contain  comments  of  a  majority  of  subjects. 


Table  4.  Content  Analysis  of  FLIR  aeene  Complexity  and  Comparison  of  FLIR  and  L3TV  Analyses. 


CRI¬ 
TERIA  TOTAL 
1st  2nd  N  SET 


DESCRIPTORS  It  KEY  WORDS 


FLIR  Analysis 
Topic  or  Sub  ject 
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3  4?  78*  saturated  than  ground)  contrast 

less  blendlng/sme&red/bluerredi 
(threat,  false  alarm) 

3.  Doclles/density  of  whlte/blobei 

2  5  7  78*  brlght/llght/heat  emitting  man¬ 

made  objects  (targets,  defenses) 

4.  Percent  area/slze/amount  of 

516  6?*  man-made  objects/ targets r  amount 

white  to  surrounds/sky 1  density 
of  man-made  per  unit  area. 

5.  Washed  out/obl Iterated  lnforaa- 
606  67%  tloni  masked  detail  1  too  white/ 

high  brightness!  poor  quality, 
over-exposed  (time  consuming). 


Number  of  man-made^  .  1.  Number  of  man-made  100* 
*  ™  objects 


objects 


Contour/edge  gradi¬ 
ent  descriptors 


White-hot-man-made 


Proportion  of  object 
area  to  surrounds^. 


Detail)  Masked  or 
washed  out 


2  4  6  67* 


1  4  5  565C 


2  1  3  33* 


6.  Defined/cultural  detail!  dlf- 
ferent/dlstlngulshable  things! 
apeclflc/complex  objects/shapes 
(differentiated  targets). 

Homogenlous/darker  background! 
lighter  trees/scene/follagei 
surrounds (distribution  of  llghti 
(masking- folia  go,  clutter) 

8.  Recognisable  patterns  1  reverse 
recognition, (man-made  geometry 
A  patterns  or  uniques  shapes) 


Cultural  detail, ob^' 
Ject  shape 


Nature  of  back-  _ 
groung  ^  T 


Patterns 


►2.  Background 


7-  Lines  and  edges 


78* 


y3.  Object  detail  1  pet-  67* 
tern,  shape,  outline 
complexity 


4.  Proportion  of  object  67* 
area  to  surrounds 


Object  detail)  lnte-  67* 
rlor  complexity 


6;_  Scene  or  Image  qua-  56* 
llty  (resolution) 


56* 


Comparison  of  FI.IR  and  L3TV  Content  Analyses.  Table  4  also  presents  a  summarization 
for  sets  in  both  content  analyses  with  solid  lines  connecting  equivalent  sets.  Simi¬ 
larities  can  be  noted  in  contrasting  the  FLIR  content  analysis  of  Table  4  with  that 
of  L3TV's  Table  2  in  that  three  sets  of  descriptors  appear  directly  comparable.  In 
both  analyses  Set  1,  mentioned  by  the  greatest  number  of  subjects,  refers  to  an  in¬ 
creasing  number  of  objects  in  a  scene  as  complexity  increases.  Similarly,  both  Sets 
4  refer  to  the  proportion  of  object  area  as  contributing  to  complexity.  Set  7  in  the 
FLIR  analysis  contains  descriptors  of  the  nature  or  appearance  of  the  background,  sim¬ 
ilar  to  Set  2  in  ^:he  L3TV  analysis.  While  5  of  the  8  sets  in  the  FLIR  analysis  have 
no  single  equivalency.  Set  2  of  the  FLIR  analysis  appears  to  be  partially  covered  by 
Sets  6  and  7  of  the  L3TV  analysis  (dashed  lines) .  Conversely,  Set  3  of  the  L3TV 
analysis  appears  partly  included  in  Sets  6  and  8  of  the  FLIR  analysis. 

From  the  preceding  discussion  and  Table  4,  it  appears  that  the  visual  parameters 
and  measures  derived  to  quantify  L3TV  displays  which  can  be  most  appropriately  ap¬ 
plied  to  FLIR  displays  are  those  based  on  L3TV  descriptor  sets  1,  2,  and  4,  namely: 


Parameters  I  and  II  a  or  b  with  derived  measures  (1)  (since  sub-structrral  objects 
were  not  mentioned) ,  (3) ,  (£)  ,  and  (5)  (since  sky  was  mentioned) . 

Evidence  presented  thus  far  indicates  that  while  there  is  some  correspondence 
between  the  visual  parameters  of  FLIR  and  L3TV  displays,  that  correspondence  is  far 
from  total.  One  way  to  evaluate  the  degree  of  correspondence  is  by  correlation.  The 
complexity  categories  assigned  the  sale  target-area  when  scaling  L3TV  and  FLIR  photo¬ 
graphs  were  correlated  for  each  subject,  the  resulting  coefficients,  r,  are  presen¬ 
ted  in  Table  5  with  an  estimate  of  the  amount  of  variance  in  one  scale  common  with  or 
due  to  the  variance  in  the  other,  r2.(27)  The  ninth  subject's  correlation  using  his 
first  FLIR  scaling  assuming  the  tactical  mission  of  reconnaissance  strike  utility  is 
presented  in  parentheses.  The  average  correlation,  Fg,  and  common  variance,  ,  ^n_ 

elude  the  ninth  subject's  data  while  the  subscript  8's  do  not. 


IWble  5.  Cwrrelrtlwna  and  flw 
Correlation  Coefficients 


ion  Variances  Between  Complexity  Scales, 
Ceaaen  Variance  Between  „ 


+.75 

+.72 

+.48 

—  — 

52* 

23* 

+.92 

r8  ‘  -7? 

85* 

*8  " 

+.70 

49* 

+.8Z 

►1 

0 

1 

3 

67* 

?2  - 

(+.31) 

(10*) 

9 

+.89 

+.89 

(*«-  B<  .01) 

79* 

79* 

55.6* 


Inspection  of  Table  5  shows  both  average  correlations  to  be  significantly  dif¬ 
ferent  from  0  at  the  99%  level  of  confidence  and  the  commong  variance  found  between 
the  two  complexity  scales  are  about  55%  and  61%.  Thus  a  moderate  linear  relation 
is  found  to  exist  between  the  L3TV  and  FLIR  complexity  scales.  These  statistics  at¬ 
tach  an  added  importance  to  the  first  two  parameters  and  4  of  their  5  measures  de¬ 
rived  earlier: 

I.  The  number  of  man-made  objects  or  object  area  in  a  scene:  (1)  The  number  of 
separate,  single  entities,  ignoring  sub-structural  aspects.  (2)  (not  applicable). 

II.  The  ratio  or  percent  of  the  area  of  man-made  objects  to  the  combined  or  se¬ 
parate  areas  of  foliace/terrain  and  sky.  (3)  The  area  of  man-made  objects.  (4)  The 
area  of  terrain/foliaqe .  (5)  Area  of  visible  skv. 

However,  the  individual  who  showed  the  greatest  mission-orientation  during  FLIR 
complexity  scaling  also  revealed  the  lowest  correlation  (+.31)  and  the  lowest  common 
variance  (10%).  That  ninth  subject's  performance  plus  the  increased  mission-orienta¬ 
tion  of  the  other  subjects  while  scaling  FLIR  scene  complexity  is  an  indication  that 
task  assignment  has  the  ability  to  modify  visual  parameters  and  perceptual  cues.  This 
is  brought  more  clearly  into  focus  when  the  ninth  subject's  two  complexity  scalings 
of  FLIR  display  photographs  are  correlated.  FLIR  complexity  values  assigned  assuming 
a  reconnaissance  strike  utility  mission  correlated  with  those  assuming  low  level  in¬ 
gress  at  r=-.13,  not  significantly  different  from  0.  When  scale  values  frem  .the  same 
subject's  second  FLIR  complexity  scaling,  the  assumed  low  level  ingress  mission^,  are 
correlated  with  his  original  L3TV  complexity  values,  a  correlation  coefficient' of 
r=-.68  results,  significantly  different  frdm  0  at  p<.01. 

Thus  although  the  50%  correspondence  found  between  the  two  complexity  scales  is 
moderately  good,  not  only  the  content  of  FLIR  displays  but  subjects’  responses  to 
that  content  differed  from  those  to  simultaneously  recorded  L3TV  displays.  Comments 
made  during  FLIR  debriefings  reveal  issues  contributing  to  the  non-correspondence 
between  FLIR  and  L3TV  scene  complexity.  Those  comments  involve:  a)  different  classi¬ 
fications  of  figures  of  interest  vs.  man-made  objects,  b)  changing  definition  of 
clutter  and  noise,  c)  increased  task/mission  orientation,  d)  target  cues  or  target 
signatures  vs.  shapes,  patterns,  and  graphic  detail,  and  e)  differing  perceptual  sets 
and  expectancies. 

The  above  indicates  th.  now  pilots  actually  use  sensor  displays,  most  particul- 
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arly  FLIR  displays  under  specific  flight  assignments,  must  be  taken  into  account  when 
designing  a  simulator  incorporating  computer  generated  imagery  for  Sensor  Operator 
Training  if  maximal  transfer-of-training  is  to  be  achieved. 

Experiment  2:  The  Matching  Study.  Because  subjects'  responses  indicated  that  scaling 
FLIR  scene  complexity  was  a  more  difficult  task,  the  matching  study  was  performed 
somewhat  as  an  empirical  check-up.  Had  a  formal  hypothesis  been  expressed  at  the  time 
of  data  collection,  it  would  have  been  that  matching  daylight  photographs  would  be 
faster  and  more  accurate  with ‘photographs  of  L3TV  displays  than  FLIR  displays  since 
L3TV  displays  seemed  to  have  more  in  common  with  daylight  views.  As  depicted  in 
Table  6 ,  performance  was  measured  both  in  terms  of  the  number  of  errors  and  the  num¬ 
ber  of  seconds  to  complete  16  matches. 


Ihhle  6.  Tlse  In  Seconds  and  Numbers  >f  Errors  Hatching  C»l»r  Ph.t.graphs. 


FLIR  DISPLAY  SCENES  LI 

ITV  DISPLAY  SCENES 

Tines  ( 

sec.  ) 

#  of 

Errors 

Tie**  ( 

sec.  ] 

I 

#  *f 

Errors 

102 

0 

14? 

0 

262 

3 

384 

4 

155 

X  -  160.0 

0 

X  - 

1.22 

276 

X  - 

227.1 

3 

X  -  2.00 

125 

4 

195 

2 

199 

B  “  65.8 

2 

8  • 

1.56 

165 

s  - 

119.2 

3 

B  -  2.19 

188 

2 

438 

6 

237 

*r  23,3 

0 

.55 

227 

®r 

35.1 

0 

T  -7? 

72 

0 

134 

0 

100 

0 

78 

0 

On  the  average,  matches  of  color  daylight  photographs  with  L3TV  display  photo¬ 
graphs  occasioned  more  errors  and  required  longer  times  than  those  with  FLIR  photo¬ 
graphs.  Because  of  high  within-group  variance,  t-tests  of  differences  between  the 
means  of  both  measures  failed  to  reach  significance.  Nevertheless,  since  the  direc¬ 
tion  of  difference  was.  opposite  to  that  which  had  been  expected,  the  implied  hypothe¬ 
sis  must  be  rejected.  Despite  the  fact  that  scaling  FLIR  scene  complexity  had  been 
cited  as  a  more  difficult  and  different  task,  the  matching  study  revealed  greater 
average  accuracy  (92.4%  vs.  85%)  with  a  shorter  average  matching  time  of  about  10 
seconds  vs.  14.2  seconds  per  picture  with  FLIR  than  with  L3TV. 

The  only  way  to  account  for  this  unexpected  reversal  is  to  observe  that  during 
the  process  of  complexity  scaling  FLIR  displays,  a  majority  of  subjects  learned  a 
visually  efficient  way  to  capitalize  on  unique  FLIR  informational  cues.  In  fact,  at 
the  end  of  his  FLIR  debriefing  one  subject  summarized,  "One  very  valid  thing  I  have 
learned  from  this  is  you  have  to  treet  FLIR  as  very  different  from  L3TV.' 


CONCLUSIONS  AND  RECOMMENDATIONS 

Conclusions. 

1.  Complexity  scaling  of  either  L3TV  or  FLIR  displays  can  identify  3  to  6  sig¬ 
nificantly  different  levels  of  complexity  for  computer  modeling  to  permit  cost-ef¬ 
fective  Sensor  Simulator  design  decisions  based  on  empirical  comparisons  of  perfor¬ 
mance  with  varying  CIG  parameters. 

2*  For  L3TV  displays,  content  analysis  revealed:  a)  Scene  complexity  was  unidi¬ 
mensional.  b)  Subjects  focused  on  real-world  man-made  objects  with  the  ground  unim¬ 
portant.  c)  A  mission  orientation  was  not  elicited,  d)  Geometries  of  the  visual  array 
dominated  most  subjects'  descriptors,  e)  Quantification  can  be  achieved  using  8 
derived  measures,  f)  "Realism"  will  be  a  function  of  these  measures,  g)  A  L3TV/CIG 
Property  List  is  possible. 

3.  For  FLIR  displays  content  analysis  revealed:  a)  Despite  intellectual  mastery, 
initial  responses  of  most  subjects  regressed  toward  expecting  a  normal  visual  scene, 
b)  The  scene-complexity  continuum  is  probably  not  unidimensional . c)  Increased  mission 
orientation  is  elicited,  d)  Perception  of  the  background  is  more  difficult  due  to 
blending,  blurring,  and  whiting-out.  e)  Some  objects  become  redefined,  f)  Scaling 
FLIR  complexity  was  more  difficult,  a  different  task,  and  a  learning  experience. 
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4.  Although  a  fair  to  moderate  linear  relation  exists  between  L3TV  and  FLIR 
scene-complexity,  there  was  no  correspondence  between  specific  sets  unique  to  each 
sensor:  Set  3  from  the  FLIR  content  analysis  (whi te=hot=man-made )  and  Set  5  from  the 
L3TV  content  analysis  (object  detail:  interior  complexity). 

5.  Results  of  the  matching  study  indicate  that:  a)  Most  mismatches  occur  with 
less  complex  scenes,  b)  Perceptual  cues  displayed  by  FLIR  sensors  are  comparable  to 
or  better  than  graphic  details  displayed  by  L3TV  sensors. 

6.  Additional  evidence  indicates  that  visual  parameters  associated  with  FLIR 
displays  are  modified  by  perceptual  set,  cognitive  factors,  and  mission  assignment. 

7.  In  general,  findings  support  a  cost-effective  CIG  simulation  of  an  L3TV  dis¬ 
play  which  stresses  the  optical  array  of  surfaces,  edges,  and  lines.  Similar  simula¬ 
tion  of  FLIR  displays  of  a  flight  over  land  is  questionable  due  to  the  nature  of  the 
sensor  and  irfluence  of  task  and  atmospheric  variables  upon  performance  and  training. 
Findings  suggest  target  acquisition  models  assuming  a  hierarchical  ordering  of  target 
detectionl,  recognition,  and  identification,  may  be  adequate  for  L3TV  but  not  FLIR. 
Recommendations . 

1.  Because  of  built-in  limitations  of  the  present  research,  conclusions  are  ten¬ 
tative  and  replication  is  recommended  with  broader  subject  and  display  sampling. 

2.  Target-areas  found  to  be  significantly  different  in  complexity  for  both  sen¬ 
sors  should  be  modeled.  The  perception  of  CIG  simulated  imagery  in  the  Sensor  Simu¬ 
lator  under  development  was  defined  by  size,  shape,  gray  shade,  contrast,  noise,  and 
curved  surface  shading.  These  should  be  systematically  varied  and  evaluated  with 
other  CIG  trade-off  parameters  as:  texture,  detail  level,  foliage  masking,  weather 
alforithm,  kind  of  noise,  and  air  speed  smearing  algorithm. 

3.  The  measures  derived  to  quantify  L3TV  displays  should  be  tested  with  the  re¬ 
commended  procedures  to  establish  a  L3TV/CIG  Properties  List  on  the  original  or  new 
stimulus  materials.  The  means  to  derive  a  FLIR/CIG  Properties  List  is  not  recom¬ 
mended  on  the  basis  of  the  present  data. 

4.  L3TV  and  FLIR  sensors  are  said  to  be  influenced  by  some  18  atmospheric  vari¬ 
ables  of  which  sun  angle  (time  of  day)  is  only  one.  To  improve  simulation  of  dis¬ 
plays  for  a  variety  of  flight  missions,  those  atmospheric  variables  need  to  be  iden¬ 
tified  and  assessed  which:  a)  have  essentially  the  same  or  similar  visual  effects  on 
sensor  displays  and  could  reduce  CIG  costs  without  sacrificing  transfer-of-training 
by  similar  programs,  and  b)  have  important  and  differing  effects  upon  sensor  dis¬ 
plays  and  which,  therefore,  must  be  included  in  ground-based  training  involving  sen¬ 
sor  simulation. 

5.  Prior  to  implementation  of  a  Sensor  Simulator,  intermediate  briefing  and 
training  materials  should  be  developed  which  address  Airmen's  expectancies  and  per¬ 
ceptual  sets  due  to  the  influence  of  cognitive  factors  and  mission  orientation  with 
FLIR  displays.  Such  materials  should  include  descriptions  and  illustrations  of  FLIR 
scenes  for  a  variety  pf  flight  missions  under  regularly  occuring  transition  periods 
and  infrequent  conditions  which  can  disrupt  operations.  A  separate  topographical 
taxonomy  may  need  to  be  developed  with  categorization  of  variables  known  or  found  to 
affect  sensor  displays. 
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COMPUTROL  -  A  New  Technique  in  Computer  Image  Generation 

Abstract 

Four  years  in  the  making,  ATS  COMPUTROL  CGI  system  has  emerged  from  its 
third  generation  with  an  approach  to  computer  imagery  that  relies  on  new  tech¬ 
niques.  The  seperation  of  three  specific  functions  into  nearly  stand  alone 
systems  has  yielded  an  array  processor  for  the  projection  analysis  which  ex¬ 
tracts  the  perspective  of  the  gaming  area  from  the  modeled  world,  an  array 
processor  for  the  visible  surface  calculation,  and  a  terminal  processor  for 
the  output  function. 

The  emphasis  has  been  on  hardware  dedication  without  the  system  being  dom¬ 
inated  by  software.  The  increase  in  speed  by  utilizing  ECL  chips,  array  pro¬ 
cessors,  cache  memory  and  specific  function  logic  has  produced  a  30-K  edge 
machine  with  a  30  cycle  update. 

This  paper  addresses  the  subject  of  edges  and  treats  the  manipulation  of 
the  modeled  world  from  points  to  lines  to  surfaces  to  edges  to  segments  as  the 
image  is  built  from  the  data  base  to  the  display. 

Parameters 

When  the  subject  of  Computer  Generated  Images  is  introduced,  the  following 
question  is  inevitable,  "how  many  edges?"  Every  visual  system  has  it  main 
standard  of  performance--television  has  its  resolution,  computer  graphics  has 
its  matrix  count,  photography  has  its  line  pairs,  printing  has  its  screen  num¬ 
ber,  and  xerography  has  its  contrast  ratio. 

A  visual  display  has  more  then  a  single  parameter  to  account  for  its 
quality.  There  are  the  factors  of  color  saturation,  contrast,  linearity,  smear 
highlight  brightness,  and  another  dozen  quantities  generated  by  the  human 
factors  people.  In  CGI,  there  is  no  stickiness  of  a  pick-up-tube,  there  is  no 
Kell  factor  for  the  random  selection  of  an  element  on  a  scan  line,  there  is  no 
smear  or  overshoot  of  video,  there  is  no  non-linearity  of  the  generated  video, 
there  is  no  color  bleeding  in  fast  motion  scenes,  and  there  is  no  Gaussian 
function  of  a  scanning  electron  beam.  What  you  generate  is  what  you  get.  The 
criterion  of  CGI  has  been,  and  is  now,  the  number  of  edges  (and  anti-aliasing, 
a  point  considered  later).  This  criterion  has  evolved  an  "edge  war". 

Is  this  "edge  war"  a  conflict  or  a  confusion?  Is  a  2000-edge  system  only 
half  the  scene  content  of  a  4000-edge  system?  Counting  edges  is  a  precarious 
game.  Which  edge  do  you  count?  There  are  the  modeled  edges  in  the  data  base, 
mathematical  lines  in  the  co-ordinate  system  connecting  points  of  the  model 
which  can  easily  exceed  20  million.  There  are  the  projected  edges  extracted 
from  the  model  of  terrain  and  targets.  The  COMPUTROL  offers  200,000  of  these 
as  wire  frames  and  polygon  edges.  There  is  the  subtraction  of  hidden  edges, 
which  in  a  good  design  are  discarded  after  the  viewpoint  is  made  so  as  not  to 
clutter  up  further  processing.  Already  we  have  a  variability  in  edge  count 
since  a  target  of  2000  modeled  edges  may  have  1000  surviving  edges  in  one 
perspective  or  may  have  100  in  another  perspective  (or  as  little  as  20,  if 
scaling  down  the  detail  with  range  reduces  the  count).  There  are  chunked  edges 
whereby  2  edges  crossing  2  edges  to  form  a  pound-sign  may  have  to  be  processed 
as  12  edges.  There  are  intersection  edges,  created  by  CGI  in  the  example  case 
of  two  9-edged  cubes  intersecting  to  make  20  edges.’  There  are  clipped  edges 
which  may  be  processed  up  to  the  point  of  border  or  depth  determination  that 
they're  out  of  the  pyramid  of  view,  and  thence  discarded.  There  are  texture 
edges  which  lend  a  realism  but  do  not  emanate  from  the  model,  and  finally 
there  are  the  visible  edges— at  last  an  edge  count  that  can  be  quantified.  But 
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the  output  edges  can  be  restricted  by  their  preceeding  operations  in  the  pro¬ 
jection  processor  or  visible  surface  processor,  where  the  time  element  in  the 
complexity  of  computations  may  be  the  limiting  factor. 

With  visible  edges  as  the  measure  of  scene  detail,  why  not  add  on  more 
processors,  larger  memory,  a  bigger  frame  buffer  and  increase  the  edge  count? 

The  design  philosphy  at  ATS  has  been  to  maximize  the  number  of  edges  that  can 
be  processed  by  three  cabinets  containing  the  function  of  the  projection  analy¬ 
sis,  the  visible  surface  determination  and  the  conversion  to  a  raster  format. 

That  is,  how  many  edges  per  225  cubic  feet  of  cabinet  space?  (This  volume  does 
not  include  the  space  for  model  data  base  or  display  components). 

It  is  the  visual  display  that  is  the  final  product.  Since  it  is  a  digital 
substratum  upon  which  the  analog  color  is  projected,  we  have  a  pixel  count,  a 
scan  line  count  and  the  color  variations  of  each  element  to  construct  the  para¬ 
meters  jf  a  computer  generated  image.  An  area  can  be  formed  by  the  maximum 
number  of  edge  crossings  per  scan  line  (approaching  the  limit  of  horizontal 
resolution)  and  the  number  of  scan  lines  (the  limit  of  vertical  resolution).  This 
area  can  be  viewed  as  a  pixel-squared  quantity  containing  the  single  frame 
output  of  a  CGI. 

The  number  of  channels  producing  a  simultaneous  output  of  different  views 
must  have  all  of  the  visible  edges  distributed  amongst  them.  That  is,  the  sum 
of  edges  in  each  output  equals  the  total  edge  count.  The  pixel-squared  area  is 
now  lengthened  along  the  horizontal  dimension  by  the  number  of  channels. 

A  determining  factor  in  edge  count  is  the  image  update  rate,  or  the  number 
of  times  a  second  the  scene  is  re-calculated  and  presented.  The  third  dimen¬ 
sion  of  time  (really,  frames  per  second  of  update),  when  stacked  on  the  area  of 
pixels  produces  a  volume  representing  the  display  limit  to  the  number  of  edges 
per  second.  Given  a  CGI  system  supporting  say,  five  channels  of  1023-line 
outputs  with  a  maximum  of  800  edge  crossings  per  scan  line,  and  a  30-cycle  up¬ 
date  rate,  we  have  a  capacity  of  the  raster  system  to  support  a  theoretical 
edge  count  of  roughly  24.5  million  edges  per  second. 

Now  to  define  an  edge  more  closely  before  examining  the  limits  imposed  by 
edge  processing  time.  In  the  C0MPUTR01  a  visible  edge  is  a  straight-line  de¬ 
fining  the  location  and  color  of  the  surface  to  its  right.  The  edge  is  defined 
by  its  height  and  top  position  (x,y)  along  with  its  slope  and  color  information. 
To  put  this  surface  on  a  raster,  it  is  broken  down  to  scan  line  segments,  each 
segment  commencing  with  an  edge  describing  the  surface  to  its  right  and  extend¬ 
ing  along  the  scan  line  until  the  next  edge  defines  another  surface.  The  seg¬ 
ment  is  constructed  by  a  scan  line  number  y,  an  intercept  point  along  the 
x-direction  of  the  scan  line,  a  delta-x  to  the  next  edge,  a  flag  to  determine 
flat  or  smooth  shading,  and  the  color  to  its  right,  in  a  simultaneous  RGB.  With 
the  pixel-raster  system  to  support  the  presentation  of  all  edges,  the  maximum 
edge  density  in  a  frame  display  is  the  edge  crossing  limit  in  the  x-direction 
and  the  scan  line  count  in  the  y-direction.  The  total  number  of  edge-crossings 
for  a  scan  line  defines  the  horizontal  resolution  of  a  CGI  frame  as  the  mini¬ 
mum  width  of  a  surface,  and  the  scan  line  count  defines  the  vertical  resolution 
as  the  minimum  height  of  a  surface. 

Comparing  the  representation  of  a  scene  with  edges  versus  the  same  scene 
represented  in  polygons,  we  find  that  polygon  surface  boundaries,  if  counted  as 
edges,  would  double  the  edge  count  when  meeting  another  surface.  That  is,  the 
three  visible  faces  of  a  cube  are  counted  as  9  edges,  but  three  surfaces  of  4 
edges  each  would  be  counted  as  12  edges.  The  accepted  ratio  is  3.67  edges  per 
polygon.  It  is  unknown  where  they  found  a  polygon  of  3.67  edges. 

As  to  the  aliasing  solution  and  the  edge  count,  the  C0MPUTR0L  in  the  F-18 
configuration  computes  eight  resolution  elements  in  the  x-direction  for  each 
output  pixel  width.  These  eight  values  are  averaged  to  form  the  color 
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components  of  the  pixel  containing  the  edge  transition.  Also  eight  times  the 
number  of  scan  lines  (8000  in  one  frame)  are  used  in  the  computation  to  average 
out  the  vertical  value  for  the  pixel  assignment.  This  computation  visually  min¬ 
imizes  the  aliasing  effect,  but  it  is  expensive  in  computation  time  and  reduces 
the  number  of  visible  edges.  The  F-18  sky-earth  generator  calculates  the  detail 
of  8000  x  8000  element  frame  and  then  mathematically  smoothes  the  discrete 
points  to  a  1000  by  1000  matrix  of  pixels. 

Before  getting  into  a  quantification  of  the  maximum  number  of  edges  which 
can  be  produced  by  C0MPUTR0L,  and  the  possibilities  and  limitations  to  this 
figure,  it  is  necessary  to  explore  the  mechanism  of  edge  processing. 

The  C0MPUTR0L  Approach 

The  creation  of  images  representing  all  possible  fields  of  view  to  a  pilot 
whose  aircraft  is  at  a  specific  attitude  over  a  specific  terrain  has  led  to  the 
development  of  a  technology  requiring  innovation  in  design  and  innovation  in 
hardware.  The  C0MPUTR0L  approach  has  been  to  develop  equipment  reflecting 
three  specific  functions  of  the  process. 

Between  the  digitized  model  residing  in  the  on-line  disks  and  the  analog 
output  of  color  video,  there  are  three  distinct  functions  of  processing.  In  a 
first  order  analogy,  and  at  a  risk  of  being  anthropomorphic,  one  could  say  that 
a  cameraman  takes  a  picture  of  the  model  as  a  wire  frame  connecting  points  in 
space,  that  this  perspective  is  passed  to  a  draftsman  who  removes  the  hidden 
lines,  and  the  final  view  is  rendered  by  an  artist  who  adds  color  and  presents 
the  view.  The  first  function  is  accomplished  by  the  projection  processor,  the 
second  function  by  the  visible  surface  processor,  and  the  third  function  by  the 
terminal  processor. 

The  logic  of  dividing  the  processing  among  three  functions  lies  with  the 
nature  of  handling  information.  The  projection  processor  receives  data  base 
points  ,  transforms  them  into  perspective  points,  calculates  color  on  the  basis 
of  polygon  surfaces  and  outputs  data  in  the  form  of  edges.  The  visible  surface 
processing  uses  the  edge  list  to  determine  the  occulting,  thence  the  structuring 
of  edge  crossings  for  sorting  on  each  resolution  line  and  outputs  encoded 
visible  edges.  The  terminal  processor  averages  the  resolution  lines  in  x,  cal¬ 
culates  and  stores  the  segments,  decodes  to  a  raster  format  and  outputs  an 
analog  signal  in  simultaneous  RGB.  If  there  is  a  change  in  requirements  of  any 
section  (say,  the  scan  line  or  frame  rate  Qr  points  of  perspective,  or  number 
of  channels)  the  change  is  made  at  the  cabinet  level  in  hardware  and  the  rest 
of  the  system  is  unaltered. 

The  first  function  of  the  projection  processor  is  to  extract  the  designated 
coverage  of  the  modeled  world  from  the  data  base  and  arrange  the  elements  of  the 
scene  into  the  perspective  of  the  viewer.  It  is  as  if  a  hypothetical  camera 
with  a  selection  of  lenses  from  narrow  field  to  wide  angle  (up  to  360°)  could 
move  to  any  location  of  the  modeled  world,  and  at  any  angle  record  with  nearly 
instantaneous  shutter  speed  the  terrain  along  with  any  selected  targets  placed 
in  their  appointed  position.  This  function  is  the  projection  processor  and 
occupies  the  first  cabinet.  The  function  is  more  complex  than  picture  taking. 
Actually,  in  the  two  dome  configuration  of  the  F/A-18,  there  are  512  perspec¬ 
tives,  each  with  a  pyramid  of  view  to  provide  accurate  mapping  to  a  curvilinear 
surface  divided  into  512  theoretical  facets.  The  co-ordinate  system  of  the 
model  (x,y,  z  location  of  points)  is  transformed  to  an  image  plane  with  the 
object  rotated,  translated  and  scaled  to  reflect  the  perspective  view.  Then 
another  transformation  places  the  image  in  the  eye  coordinate  system  with  the 
intervening  "window"  (the  viewed  display)  receiving  it’s  projection  of  the 
scene. 
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The  color  information  in  the  modeled  data  base  contains  the  polygon  surface 
normals,  optical  properties,  intrinsic  color,  and  a  flag  for  flat  or  curved  sur¬ 
face  shading.  Since  edges  contain  the  color  to  their  right,  a  visibility  and 
sun  direction  must  be  calculated  in  the  projection  processor.  The  combination 
of  the  surface  normals,  their  reflectivity,  the  sun  direction,  the  visibility, 
the  modeled  color,  the  sun  color  and  the  gradient  shading  can  be  used  to  assign 

the  color  component  to  the  edge,  depending  on  its  orientation  in  the  viewed 

space. 

The  polygons  are  decoded  into  edges  (since  this  is  the  medium  of  processing) 
and  now  the  geometrical  points  can  be  addressed  to  determine  if  they  fall  into 
the  viewing  window.  The  edges  at  the  borders  of  the  window  are  clipped  and, 
with  the  Z-direction  component  known,  the  edges  falling  on  the  viewer's  side 
of  the  window  are  also  clipped.  The  surviving  edges  receive  computations  to  de¬ 
termine  their  height,  their  slope  and  their  depth  gradient.  With  this  attendent 
information,  the  edges  are  assembled  and  sorted  on  the  basis  of  the  y-top  (that 

is,  the  hightest  top  point  of  an  edge).  This  data  is  buffered  for  use  by  the 

second  function,  the  visible  surface  processor. 

All  the  edges  in  the  perspective  scene  are  placed  in  the  main  memory  in 
y-sorted  order.  The  visible  surface  processor  searches  its  own  "current  edge" 
buffer  of  previous  scan  lines  for  new  edges  which  are  sent  to  a  bucket  sort  of 
the  x-intercepts.  Now  in  x-order,  an  occulting  algorithm  is  invoked  to  ascer¬ 
tain  the  overlay.  The  winning  edges  at  each  x- intercept  point  are  added  to  the 
current  edge  list.  If  a  new  edge  is  required  by  the  intersection  of  two  sur¬ 
faces,  it  is  created  at  this  time  and  placed  with  the  current  edges. 

An  occulting  or  Z-clipped  fog  can  be  generated  by  the  visible  surface  pro¬ 
cessor:  the  general  fog  is  created  in  the  projection  processor  as  a  visibility 
restriction  to  all  objects,  but  a  Z-clipped  fog  starts  at  a  scene  depth  like  a 
fog  patch,  and  therefore  it  is  treated  as  an  overlay  of  a  transluscent  surface. 

During  the  second  stage  of  x-sorting,  color  data  for  terminating  scan  line 
segments  are  calculated.  The  end  point  colors  are  used  in  the  smooth  shading 
algorithm. 

The  visible  edges  are  assembled  in  blocks  of  1024  intercepts  and  stored  in 
two  frame  buffers  in  which  each  can  hold  64  thousand  visible  edges  to  be  dis¬ 
tributed  across  all  the  output  channels. 

A  detection  of  impending  image  overload  (that  is,  approaching  maximum 
intercepts  in  the  buffer  or  the  maximum  16.7  millisecond  computation  time  for  a 
frame)  is  monitored  by  the  CPU. 

The  encoded  visible  edges  in  the  frame  buffer  are  formatted  in  89  bits  as 
follows: 


FUNCTION  BIT  REPRESENTATION 

Color  (R,G,B) ,  C  (at  top)  21  (7  bits  each  color) 

Color  Gradient,  G  (along  edge)  27  (9  bits  each  color) 

Top  Scan  Line  Position,  Y  9 

Reciprocal  Slope,  S  12 

Edge  Height,  H  9 

Position  of  X  (top)  10 

Smooth,  Shade  Flag,  F  1 

The  terminal  processor  receives  data  in  the  above  format  and  derives  an 
edge-to-segment  calculation  to  transcribe  the  edges  into  their  representation 
on  scan  lines.  This  process  (1)  calculates  the  x-intercept  and  color  intercept 
of  each  edge:  (2)  generates  a  segment  width  of  delta-x,  a  delta  color,  C,  and 
smooth  shade  flag,  F:  (3)  deletes  segments  of  zero  width:  and  (4)  stores  the 
results  in  a  ping-pong  buffer  cache.  This  buffer  is  a  sequential  wrap-around 
write  and  serial  read  memory. 
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The  smooth  shader  and  raster  generator  decodes  the  read  side  of  the  ping- 
pong  buffer  data  on  a  scan  line  time  basis  and  digitally  produces  three  colors 
at  a  16  nano-second  rate  (2000  units  in  x  for  each  channel).  The  D/A  conver¬ 
ters  (three  for  each  output  channel)  complete  the  transformation  to  video  of 
simultaneous  red,  green,  and  blue  to  the  color  display  system. 

The  Architecture  of  C0MPUTR0L 

Intrinsic  in  the  design  philosophy  of  generating  the  maximum  number  of 
edges  in  three  cabinets  is  the  special  purpose  processor.  To  incorporate  the 
peak  effeciency  for  a  specific  function  in  digital  processing,  a  custom  design 
had  to  be  developed.  To  wit,  there  has  been  no  general  purpose  computer  and  its 
program  which  could  play  master  chess  until  a  special  purpose  processor  was 
designed. 

The  three  functions  each  have  an  array  processor.  This  pipelining  of  the 
computation  requires  dedicated  hardware  arranged  so  that  each  processor  does  one 
operation  of  the  general  algorithm.  A  string  of  these  processors  then  outputs 
a  specific  function--like  a  matrix  transformation--in  each  cycle  time.  Because 
a  function  is  repetitive,  like  the  scaling,  rotation  and  translation  of  a  point 
or  the  depth  determination  of  an  edge,  the  operation  on  data  requires  no  system 
commands.  With  the  increased  speed  comes  the  need  for  short  data  paths,  and 
with  the  specific  format  and  content  of  each  piece  of  information,  the  memory 
cache  is  utilized.  These  are  local  memory  units,  located  in  the  same  cabinet  as 
the  array  processor  and  serving  the  input/output  storage  needs.  Simplification 
of  the  addressing  of  memory  results.  That  is,  a  place  for  each  datum  and  each 
datum  in  its  place--the  epitome  of  organization. 

As  developed  in  the  discussion  of  the  function  in  each  processor,  each 
array  handles  information  in  a  different  form.  The  projection  array  processor 
does  matrix  transformations  and  operates  on  floating  point  numbers  from  the 
data  base  model.  The  visible  surface  array  processor  does  a  computation  for 
overlay  (depth  of  an  edge)  and  operates  with  a  fixed  point  number.  The  ter¬ 
minal  array  processor  decodes  a  formatted  data  word  and  stores  scan  line  seg¬ 
ments  in  a  frame  buffer.  In  each  case  an  array  processor  is  responsible  for  a 
unique  function  and  does  its  own  thing  without  a  program  command,  or  an  inter- 
upt  or  a  wait  time  for  main  memory  addressing. 

The  fact  that  C0MPUTR0L,  with  its  pipelined  array  processors,  is  not  soft¬ 
ware  dependent  introduces  a  new  concept  to  a  digital  computation.  The  task  of 
the  CPU  is  to  manage  main  memory  and  provide  timing  signals  to  the  processors. 

If  a  change  in  system  parameters  is  mandated,  this  usually  results  in  a  board 
swap  or  a  hardware  alteration.  A  lengthy  program  patch  and  software  modifi¬ 
cation  is  not  required. 

Now  that  the  functions  and  the  methodology  of  the  computer  generated  image 
have  been  touched  upon,  the  subject  can  turn  to  the  quantitative  question,  "yes, 
but  how  many  edges?" 

The  limit  on  processing  for  the  current  system  is  not  a  fixed  and  absolute 
number,  but  is  dependent  on  the  scene  composition. 

In  the  model,  a  nearly  unbounded  area  can  be  frozen  in  the  data  base.  A 
600  by  600  nautical  mile  area  with  detailed  terrain  and  extensive  target  models 
may  amount  to  300  megabytes,  placed  on  ten  discs.  The  selected  area  in  the 
total  field  of  view  (a  composite  of  the  sum  of  possible  instantaneaous  fields 
of  view)  may  comprise  400,000  points  when  placed  in  main  memory.  The  projection 
processor  is  limited  by  the  y-sort  memory  (that's  the  assembly  of  edges  for  its 
output)  to  64-k  edges.  These  are  the  wire-frame  edges,  the  total  for  all  chan¬ 
nels,  already  clipped  for  the  pyramid  of  view,  but  containing  hidden  edges  which 
later  may  be  obliterated.  The  visible  surface  processor  has  a  limit  due  to  the 
holdings  of  the  current  edge  buffer,  and  its  work  load  depends  on  number  of 


edges,  the  crossings  per  scan  line,  and  the  overlays  per  crossing.  The  ter¬ 
minal  processer  has  two  64-k  edge  frame  buffers  to  accommodate  the  odd  and  even 
fields,  and  this  capacity  is  therefore  dependent  on  frame  time.  The  theoretical 
upper  limit  associated  with  a  30-cycle  update  system  would  be  64-k  visible  edges. 
However,  the  overall  edge  limitation  lies  in  the  scene  complexity,  and  for  an 
average  view  of  terrain  and  a  few  targets,  the  number  of  visible  edges  can  be 
taken  as  30  thousand. 

Summary 

It  has  been  shown  how  three  cabinets  can  produce  a  picture  worth  30- thou¬ 
sand  edges.  The  packing  density  of  140  edges/cubic  foot  is  yet  to  be  increased 
in  the  fourth  generation  of  CGI.  But,  as  to  the  moment,  C0MPUTR0L  can  supply 
to,  say,  five  channels  in  a  canopy  of  virtual  image  lenses,  or  to  a  5-projector 
dome  configuration,  about  700  cubes  in  each  projector,  or  roughly  the  limit  of 
the  resolution  of  the  display  device. 
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Pixels 
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Limiting  Factor 
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2.4  M-Byte  (Ave.  Edge 
is  12.5  Bytes) 


18  nSec/edge  per  pass. 

#  Passes  depends  on 
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&  overlap  surfaces/ 
crossing 
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Figure  3 
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A  CONTINUOUS  WIDE-ANGLE  VISUAL  SYSTEM  USING  SCANNED  LASERS 


ABSTRACT 


A  wide-angle  high  resolution  visual  system  is  described 
which  is  specifically  designed  for  the  helicopter  nap- 
of-the-earth  flight  training  role,  where  good  depth  of 
focus  and  high  scene  detail  are  required. 


INTRODUCTION 


The  need  to  develop  a  wide-angle  visual  system  having  the  capacity  for 
producing  a  very  detailed  scene  over  the  whole  field  was  originally 
considered  by  RSL  more  than  ten  years  ago.  The  training  application 
was  at  that  time,  and  still  is  today,  the  low-level  mission  pilot  in 
fixed  wing  or  rotary  wing  aircraft.  Typical  mission  programmes  for 
these  aircraft  types  are  fairly  well  known  and  need  not  be  described 
here1  >7 .  However ,  there  are  perhaps  three  primary  aspects  of  such 
missions  which  can  be  usefully  considered  separately. 

Firstly,  the  take-off  and  flight  to  a  pre-determined  position  during 
which  the  aircraft  will  descend  to  contour  flight  and  finally,  in  the 
case  of  a  helicopter,  to  NOE  flight. 

As  low-level  flight  is  potentially  very  hazardous,  there  would  be  great 
advantage  in  a  pilot  gaining  as  much  experience  as  possible  of  contour 
and  NOE  flight  on  a  simulator,  both  before  and  during  real  flight 
exercises.  Indeed,  some  flight  patterns  and  certain  manoeuvres 
considered  essential  under  real  battle  conditions,  may  not  be  learnt 
safely  in  the  aircraft.  In  addition,  the  experienced  pilot  could  be 
checked  out  thoroughly  and  more  conveniently  on  a  low-level  flight 
simulator. 

The  visual  display  for  such  a  simulator  would  require  a  FOV  of  at  least 
120°  horizontal  by  60°  vertical,  and  a  scene  rich  in  content,  texture, 
colour,  and  structural  detail.  Resolution  should  be  preferably  as  high 
as  practicable,  but  need  not  approach  human  eye  performance. 

The  second  aspect  to  be  considered  is  reconnaissance  and  target  acquisi 
tion.  True  target  identification  training  may  require  real  world  condi 
tions,  although  even  for  this  purpose  some  improvisation  should  be 
possible.  For  instance,  objects  may  be  artificially  moved  closer,  or 
made  larger  in  a  visual  system  in  order  to  compensate  for  reduced 
resolution,  without  loss  of  training  value.  Where  telescopes  or 
infra-red  vision  aids  are  used  however,  a  secondary  image  generator 
could  be  Incorporated  to  provide  fairly  realistic  ’magnified’  direct 
view  or  head-up  images.  It  may  not  be  essential  for  a  ground  target 
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to  be  seen  to  move  in  every  case,  but  clearly  a  variety  of  target 
positions  would  be  needed.  Aerial  targets  may  also  have  to  be 
catered  for. 

The  third  aspect  is  the  weapon  aiming  and  delivery  phase  of  a  mission, 
if  applicable.  The  chief  requirements  for  simulation  here  are  to  have 
good  correlation  between  aircraft  pointing  attitude  and  the  displayed 
target  and  a  reliable  method  of  measuring  fire  accuracy.  In  addition, 
the  display  of  weapon  traces,  weapon  strikes,  smoke,  dust,  etc,  would 
be  desirable. 

It  is  worth  summarising  the  performance  of  current  visual  systems  in 
relation  to  the  needs  of  the  low-level  flight  mission. 

TV  camera-model  techniques 

The  terrain  modelboard  provides  very  high  scene  detail,  but  the  resolu¬ 
tion  obtained  from  a  TV  camera  is  only  10  arc  min/OLP  at  best  and 
degrades  to  much  less  than  this  in  the  near  scene  due  to  limited  depth 
of  focus  in  the  optical  probe.  This  effect  is  accentuated  in  the  wider 
FOV  probes  (120°H  x  6o°V)  which  also  suffer  closeness  of  approach 
limitations,  both  horizontally  and  vertically.  Tilt  focus  has  been 
used  in  an  attempt  to  improve  the  apparent  depth  of  focus,  but  is  only 
applicable  to  relatively  flat  terrain  such  as  air  fields  or  general 
terrain  viewed  from  high  altitudes.  Thus  in  low-level  flight  where 
it  is  most  important  to  see  close  three-dimensional  objects,  conventional 
TV  probe  systems  are  least  effective.  Moving  targets  and  weapon  effects 
would  need  to  be  superimposed  CGI. 

Computer  generated  imagery 

FOV,  resolution  and  gaming  area  are  not  limited  for  any  fundamental 
reason  in  CGI  and  the  whole  scene  is  always  in  focus.  FOV  can  be 
made  large  by  using  multiple  channels,  and  resolution  should  be 
adequate  if  suitable  display  devices  are  used.  However,  the  basic 
problem  is  in  providing  sufficient  scene  detail  for  terrain  flight 
over  a  large  area  and  maintaining  a  sufficient  level  of  detail  as 
various  terrain  features  are  approached.  Systems  using  shading  and 
texture  have  been  produced,  but  severe  problems  have  been  encountered 
in  the  development  of  practical  systems  for  low-level  flight. 

In  addition  the  cost  of  development  and  purchase  of  an  adequate  CGI 
system  threatens  to  be  prohibitive. 

Display  devices 

Increasingly  it  is  becoming  accepted  that  a  projected  display  has 
several  significant  advantages  over  collimated  CRT  displays.  For 
example,  it  allows  a  cab  change  to  be  made  more  easily  and  a  variety 
of  cab  configurations  to  be  used.  Of  particular  importance  for  very 
wide-angle  displays  is  the  potential  for  pitching  and  rolling  the 
displayed  field  such  that  the  horizon  and  the  terrain  image  remain 
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visible  to  the  pilot  at  any  roll  angle,  and  throughout  a  large  pitch 
range. 

With  current  projectors  there  are  usually  significant  penalties  to 
pay  for  these  advantages,  such  as  increased  cost  and  complexity  of 
hardware,  poorer  resolution,  increased  colour  misregistration,  and 
often,  reduced  brightness.  For  multi-projector  arrays  there  is 
the  added  problem  of  edge  and  colour  matching,  and  the  consequent 
high  maintenance  costs. 


RSL  Wide-Angle  Scanned  Laser  Visual  System 


It  is  not  difficult  to  specify  a  visual  system  that  would  be 
completely  satisfactory  for  all  low-level  flight  training 
applications.  It  would  in  effect  have  a  performance  approaching 
real  world  conditions  in  many  basic  parameters.  For  instance: 


FOV  -  Limited  only  by  aircraft  structure, 

say  240°H  x  90°V 

Resolution  -  1-2  arc  minutes 

Scene  content  -  Realistic  detail  and  texture  on  all 

features,  trees,  ground  buildings, 
pylons,  wires,  etc 

Time  of  day/weather  -  Fully  variable 

Targets /weapon  effects  -  Moving  vehicles  in  variable  forma¬ 
tions,  accurate  weapon  trajectories, 
exploding  shells,  etc 


Obviously  current  technology  and  many  practical  limitations,  including 
cost,  prevent  this  ideal  from  being  achieved,  however  the  current 
development  of  the  RSL  Wide-Angle  Visual  System  is  aimed  at  achieving 
a  substantial  step  toward  this  ideal. 

The  system  employs  TV  raster  principles,  but  through  the  use  of  laser 
beams  and  novel  scanning  methods,  the  usual  inherent  limitations 
imposed  by  lenses  on  FOV  and  resolution  are  avoided2. 
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The  target  specification  for  the  system  is  as  follows: 


FOV 

Raster 

Refresh  rate 
Scene  content 

Resolution  (limiting) 

Effective  depth  of  field 
Colour 

Minimum  eye  height 
Horizontal  clearance 
Motions  - 

Heading 

Pitch 

Roll 

Gaining  area 


175°  horizontal  x  60°  vertical,  continuous 
5,280  vertical  lines/frame 
60  fields/sec  interlaced 

Approximately  1200  pixels/line  and  6.25 x  10^ 
pixels/frame 

Vertical  3.5  arc  minutes.  Horizontal  4.0 
arc  minutes 

5  mms  -  5  metres 

Red,  green,  and  blue,  separately  modulated 
1.8  mms 
5  mms 

Accel  200°/sec2;  rate  180°/sec; 
continuous  360° 

Accel  150°/sec2;  rate  50°/sec; 
range  to  ±85° 

Accel  350°/sec2,*  rate  120°/sec; 
continuous  360° 

As  required  -  typically  13  Km  x  5  Km 
for  1000:1  scale 


Display  brightness 

Modulation  depth 

Moving  targets) 
Weapon  effects) 


5  fL 
>  50:1 

Superimposed  CGI 


The  system  specified  therefore  will  produce  a  large  FOV  from  a  single 
projector  giving  little  image  distortion  and  a  very  high  scene  content 
The  image  being  continuous  will  be  free  from  the  usual  colour  and  edge 
discontinuities  to  which  multi-channel  displays  are  prone.  The  camera 
can  provide  a  minimum  pilot's  eye  height  of  6  feet  at  1000:1  scale, 
and  an  electro-optic  fast  focussing  device  produces  an  effective  depth 
of  field  from  5  mms  to  the  far  distance. 


Resolution  is  approximately  twice  that  of  CCTV  systems  and  with  the 
use  of  the  focussing  device,  the  very  near  scene  will  show  a  level 
of  detail  and  texture  that  is  probably  unobtainable  on  any  other 
system. 

The  specification,  as  stated,  will  therefore  very  comfortably  meet 
the  need  for  a  detailed  scene  which  should  enable  the  pilot  to 
adequately  carry  out  low-level  flight  training. 

Full  mission  training  capability  would  demand  the  simulation  of 
moving  targets,  sights,  night  vision,  weapon  effects,  etc. 

Unmagnified  night  vision  and  direct  view  optical  systems  could  be 
simulated  by  using  a  small  part  of  the  wide-angle  scene  as  defined 
by  the  required  look  direction,  or  with  magnified  targets  being 
produced  by  a  separate  image  source.  These  images  could  be  two- 
dimensional  only  against  a  suitable  background,  in  most  cases,  and 
might  be  CGI  or  photographs  stored  on  video  discs.  A  more  difficult 
aspect  to  achieve  would  be  the  creation  of  moving  targets  and  weapons 
within  the  wide-angle  scene.  Schemes  for  superimposing  CGI  on  model- 
board  scenes  have  been  studied  and  some  possibilities  identified. 

The  display  might,  for  example,  be  divided  on  a  range  basis  between 
a  CGI  scene  and  a  modelboard  scene,  with  any  near  scene  video 
overriding  the  more  distant  CGI.  Obscuration  of  CGI  targets  by  near 
scene  features  would  thus  be  achieved. 

However,  in  most  cases,  targets  and  weapons  are  observed  only  through 
head  down  vision  aids  and  therefore  it  would  probably  not  be  cost 
effective  to  simulate  the  dynamic  features  of  a  battlefield  environ¬ 
ment  for  the  'out  of  the  window'  scene. 

LASERS  IN  VISUAL  SIMULATION 

As  is  well  known,  lasers  have  many  possible  uses  in  science  and  engin¬ 
eering  and  the  name  laser  has  now  become  associated  with  much  'state 
of  the  art'  technology,  such  as  nuclear  fusion,  holography,  ranging, 
weapon  aiming,  and  data  recording  and  transmission.  Against  this 
perhaps  confusing  background,  it  may  be  helpful  to  outline  the 
reasons  for  using  lasers  in  the  RSL  visual  system  and  consider  the 
main  implications  of  operating  these  devices. 

The  laser  is  able  to  emit  an  almost  parallel  beam  of  light  of  high 
power  density  which  can  be  put  through  almost  any  chosen  aperture. 

If  a  conventional  light  source,  such  as  an  incandescent  lamp  were 
used  for  example,  only  a  fraction  of  the  total  light  could  be 
directed  into  a  narrow  beam. 
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In  the  laser  camera,  this  narrow  beam  allows  a  very  good  compromise 
between  resolution  limit  and  depth  of  field  to  be  achieved  with  the 
added  possibility  of  varying  the  beam  convergence  dynamically  to  give 
optimum  focus  over  a  very  wide  object  range.  By  comparison,  in  a 
conventional  probe  and  television  camera  system,  the  aperture  size  is 
normally  sized  to  retain  picture  quality  and  therefore  typically, 
produces  a  small  depth  of  focus.  Also  focus  changes  must  operate  on 
the  whole  image  at  once  so  that  no  allowance  for  variation  in  scene 
depth  can  be  made .  In  the  case  of  a  projector  where  light  is  always 
at  a  premium,  the  laser  can  provide  increased  power  without  causing 
a  change  in  spot  size  and  consequent  loss  of  image  quality  that  is 
typical  of  CRT  devices.  The  laser  is  in  effect  a  very  efficient 
light  source  for  the  type  of  imaging  systems  required  in  simulation. 

Colour 


The  RSL  laser  visual  system,  from  initial  conception,  has  been 
designed  to  provide  a  full  colour  display.  Colour  is  generated 
by  red,  green,  and  blue  laser  beams  which  are  combined  and  scanned 
as  a  single  beam  and  then  separated.  Single  lasers  do  exist  that 
provide  the  three  primary  colours,  but  a  high  conversion  efficiency 
is  possible  using  an  argon  ion  laser  for  green  and  blue  and  a  krypton 
ion  laser  for  red.  Unlike  the  phosphors  used  on  CRT's,  laser  emission 
lines  are  highly  monochromatic  and  spectrally  pure,  but  the  power 
available  in  different  lines  varies  widely  and  also  some  lines  do  not 
occur  at  convenient  wavelengths. 

The  three  most  appropriate  lines  are  476  nm  (blue) ,  514  nm  (green)  and 
647  nm  (red) .  The  chromaticity  diagram  shows  that  the  triangle  defined 
by  these  three  wavelengths  includes  the  chromaticities  of  most  natural 
features  and  is  considerably  larger  than  the  triangle  for  the  phosphors 
of  colour  television  tubes.  However  for  this  colour  potential  to  be 
achieved  in  practice,  a  suitable  distribution  of  power  between  the 
three  laser  lines  must  be  achieved.  In  particular,  the  low  luminosity 
of  the  red  line  must  be  compensated  for  by  using  higher  power  in 
krypton  than  in  argon,  or  alternatively  a  proportion  of  the  argon  light 
may  be  employed  to  produce  red  by  exciting  fluorescence  within  a  suitable 
dye. 

Safety 

The  type  of  laser  employed  in  the  described  system  will  have  an  output 
power  in  the  order  of  several  watts  and  is  described  as  a  Class  IV 
laser.  If  a  stationary  unmodified  beam  from  such  a  laser  entered  the 
eye  directly,  permanent  damage  to  the  retina  could  result.  This  is 
the  chief  potential  hazard  from  these  devices,  although  only  a  very 
few  such  accidents  have  been  recorded. 
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The  laser  visual  system  described  will  incorporate  a  number  of  features 
which  will  provide  complete  safety  for  the  user.  In  brief,  these  will 
be  as  follows: 

1.  The  beam  emerging  from  both  camera  and  projector  is  not 
stationary,  but  is  in  fact  scanned  rapidly  over  large 
angles.  The  pupil  of  the  eye,  even  if  quite  close  to 
the  frame  scanner,  subtends  a  very  small  angle  compared 
to  the  total  scan  angle  and  therefore  only  receives 
incident  illumination  for  a  fraction  of  the  scan  period. 

The  averaging  effect  produced  effectively  reduces  the 
intensity  of  the  laser  beam  by  a  large  factor.  Should 
scanning  begin  to  slow  for  any  reason,  the  lasers  will 
be  rapidly  switched  off.  The  inertia  of  the  scanning 
systems  does  not  allow  instantaneous  loss  of  scanning. 

2.  The  observer  of  the  display  will  be  prevented  by  suitable 
masking  and  structure,  from  placing  his  eye  within  the 
volume  that  is  scanned  by  the  laser  beam  (between  the 
projector  and  the  screen) .  This  will  apply  both  inside 
and  outside  the  fuselage  structure.  Only  the  reflected 
light  from  the  screen  will  be  visible  and  even  if  the 
laser  spot  was  stationary  on  the  screen,  the  intensity 

at  the  observer's  eye  point  would  be  several  orders  below 
the  damage  threshold. 

3.  It  is  a  basic  system  requirement  that  the  laser  camera  be 
located  in  a  light  tight  enclosure.  By  incorporating  door 
interlocks  the  area  will  also  provide  laser  safety. 

4.  Maintenance  and  adjustment  of  the  system  will  be  carried 

out  using  eye  safe  laser  power  levels  of  the  order  of  1  milli¬ 
watt.  Power  reducing  filters  could  be  automatically 
inserted  on  removal  of  safety  covers. 

5.  The  team  involved  in  the  development  of  the  scanned  laser 
system  are  being  monitored  at  regular  intervals  by  Dr  Brennan 
of  the  Institute  of  Aviation  Medicine 

SYSTEM  CONCEPT 

Introduction 

The  system,  as  shown  in  Figure  1,  is  based  upon  simultaneously  scanning 
a  pair  of  tri-colour  light  spots,  one  over  a  modelboard  and  the  other 
over  a  front  projection  screen:  A  video  signal  is  produced  by  collec¬ 
ting  the  light  instantaneously  scattered  from  the  model  surface  on  a 
photo-electric  detector,  and  using  the  resulting  voltage  to  modulate 
the  projected  spot  to  form  an  image. 


During  the  conceptual  phase  a  number  of  decisions  were  made 
related  to  field  of  view  format  orientation,  raster  orientation, 
collimated  or  non-collimated  display,  etc.  Some  of  these  are 
discussed  below: 

Field  of  view  orientation 


Flight  simulator  visual  systems  usually  have  the  field  of  view 
defined  with  respect  to  aircraft  axes.  In  a  wide-angle  system 
where  the  ratio  of  picture  width  to  picture  height  is  large, 
this  results  in  a  large  variation  in,  for  example,  the  length 
of  the  horizon  line.  In  a  low-level  role,  where  the  use  of 
terrain  features  for  concealment  is  important,  this  can  result 
in  an  obstruction  at  the  edge  of  the  field  of  view  disappearing 
from  view  as  the  pilot  turns.  The  provision  of  an  horizon 
stabilised  picture  prevents  this  loss  of  information  and  studies 
have  shown  that  the  provision  of  an  horizon  stabilised  display 
120°  wide  by  60°  high  permits  obstacle  avoidance*. 

From  the  practical  point  of  view  the  implementation  of  an  horizon 
stabilised  display  permits  the  inclusion  of  the  pitch  and  roll 
mechanism  at  the  projector  where  there  is  more  room  than  at  the 
camera  probe. 

Line  scan  orientation 


A  wide-angle  system  can  be  designed  using  lenses  of  moderate  field 
angles  by  arranging  the  frame  scanner  to  rotate  as  shown  in  Figure 
2.  The  maximum  field  angle  required  in  such  a  system  is  that  of 
the  line  scan  which  in  this  case  is  60°.  If  the  projection  lens 
is  rotated  with  the  frame  scanner,  then  each  scan  line  follows 
exactly  the  same  path  through  the  projection  lens  and  frame  scanner 
and  so  gives  uniformity  of  resolution  across  the  scan  lines.  The 
adoption  of  this  approach  in  the  system  developed  has  resulted  in 
vertical  scan  lines  which  enables  the  apparent  depth  of  field  of  the 
camera  to  be  improved. 

Display  screen 

A  wide-angle  picture  produced  by  a  single  projector  is  conveniently 
projected  onto  the  inner  surface  of  a  hemispherical  screen  from  the 
centre  of  radius.  In  the  system  developed,  this  screen  is  viewed 
directly,  and,  due  to  the  very  large  field  angles  subtended  at  the 
viewer's  eyes,  gives  a  high  degree  of  subjective  realism  and  depth 
perception . 

System  description 

The  technique  employed  is  essentially  a  form  of  CCTV  in  that  scanning 
is  used  to  generate  and  display  the  image.  The  method  of  producing 
the  camera  and  projector  raster  scans  is  by  deflection  of  laser  beams 
using  rotating  mirrors  which  are  very  accurately  synchronised.  This 
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general  approach  has  been  in  use  for  many  years  in  film  recording 
applications,  but  has  previously  been  limited  to  broadcast  televi¬ 
sion  scanning  standards  and  narrow  field  of  view  format. 

In  this  case  a  vertical  line  scan  was  chosen  operating  in  synchronism 
with  a  continuously  rotating  frame  scan  optical  system.  The  line 
scanner  produces  a  12°  scan  which  is  expanded  to  60°  by  the  optical 
system,  with  successive  line  scans  rotated  about  the  vertical  axis  by 
the  frame  scan  prisms  to  give  a  total  horizontal  angle  of  180°  as 
illustrated  in  Figure  2.  In  this  way  the  use  of  very  wide-angle  optics 
is  avoided  together  with  the  potentially  severe  aberrations  which  are 
normally  associated  with  them.  In  fact  the  image  quality  does  remain 
consistent  throughout  the  horizontal  field.  In  addition,  the  scan 
mechanisms  and  associated  optics  in  each  unit  can  be  similar  or  identical, 
because  both  the  camera  and  projector  generate  a  raster  scan. 

When  projected  onto  a  terrain  model  the  scan  pattern  emitted  by  the 
camera  is  seen  to  be  radial  and  originates  from  a  point  immediately 
below  the  centre  line  of  the  probe. 

The  raster  light  is  scattered  from  the  model  surface  and  detected  by  a 
number  of  photomultipliers  arranged  in  a  similar  way  as  lamps  would  be 
in  order  to  provide  uniform  illumination.  However,  the  exit  pupil  of 
the  camera,  which  emits  the  laser  beam  constitutes  the  point  of  perspec¬ 
tive  or  look  point  as  does  the  entrance  pupil  of  a  conventional  camera. 

The  projector  scans  an  identical  raster  onto  a  hemispherical  screen, 
which  is  intensity  modulated  by  an  acousto-optic  device  that  is  driven 
by  the  video  signals  obtained  from  the  photomultipliers.  An  undistorted 
image,  in  correct  perspective,  is  then  formed. 

In  producing  a  display  in  full  colour,  two  ion  lasers  will  be  used  in  the 
camera  and  two  in  the  projector.  An  argon  laser  will  provide  blue  and 
green  emission  lines  and  a  krypton  laser,  red.  In  the  camera  these  three 
beams  will  be  combined  prior  to  scanning  and  then  detected  by  the  photo¬ 
multiplier  triads,  each  having  a  detector  independently  sensitive  to 
red,  green  and  blue  light.  In  the  projector  the  three  beams  will  be 
separately  modulated  by  three  acousto-optic  cells  driven  by  the  corres¬ 
ponding  colour  video  signal  and  then  combined  for  scanning  as  in  the 
camera . 

Production  of  the  raster  scan 

In  order  to  achieve  high  definition  over  180°  of  horizontal  field  (line 
scan  vertical) ,  a  much  larger  number  of  scan  lines  are  required  than  in 
broadcast  television  systems. 

A  twelve  facetted  mirror  polygon,  operating  in  conjunction  with  an  array 
of  10  roof  prisms,  produces  120  lines/revolution  of  the  polygon. 

Rotational  frequency  is  1320  Hz  (79,200  rpm)  giving  a  line  rate  of 
158,400  per  second. 
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The  polygon  of  Beryllium  is  an  integral  part  of  a  motor  which  is 
supported  on  air  bearings  and  is  powered  by  a  three-phase  synchronous 
drive  system. 

The  frame  scanner  consists  of  three  independent  rotating  optical 
systems  which  operate  downstream  of  the  line  scanner. 

The  pupil  switch  plate  moves  the  exit  pupil  of  the  system  'rom  one 
frame  scan  to  the  other,  every  half  revolution  of  the  frame  scan  tube. 

In  this  way,  each  frame  scan  prism  projects  alternate  fields  into  the 
same  18o°  sector.  Successive  fields  are  displaced  slightly  to  provide 
interlace.  The  de-rotation  prism  rotates  at  precisely  half  the  speed 
of  the  frame  scan  tube  to  prevent  the  line  scan  from  rotating  within 
the  raster. 

The  frame  rate  was  selected  to  be  30  per  second  which  minimises  flicker 
and  allows  5,280  lines  per  frame.  This  line  density  is  equivalent  to 
less  than  4  arc  minutes  per  line  pair. 

Synchronisation  of  the  camera  and  projector  scanning  systems  is  controlled 
by  a  common  pulse  generator  that  limits  line  and  frame  jitter  to  a  level 
which  is  not  detectable  by  the  viewer. 

Wide  bandwidth  video  system 

To  achieve  the  high  picture  content  that  is  considered  essential  for  low 
level  flight  training  over  the  large  field  angle  available,  a  particularly 
wide  video  bandwidth  system  is  required,  compared  with  the  5.5  MHz  currently 
used  on  625  line  TV.  The  system  bandwidth  chosen  was  from  DC  to  100  MHz. 

This  means  that  the  constituent  components  (ie,  photomultipliers,  head 
amplifiers,  mixers,  video  channel,  and  finally  the  acousto-optic  modulator) 
must  have  responses  individually  in  excess  of  this  bandwidth.  The  advantage 
of  this  exceptionally  wide  bandwidth  video  system  is  that  6.25  million  picture 
points  per  frame  are  produced.  This  gives  approximately  twice  the  resolution 
of  conventional  CCTV  systems,  but  of  course  over  a  much  larger  field  of  view. 

IMAGE  GENERATION 

The  image  displayed  by  the  projector  can  be  derived  from  a  modelboard,  from 
computer  generated  imagery,  or  a  combination  of  both. 

Modelboard  with  laser  camera 


The  chosen  scale  for  a  NOE  modelboard  is  1000:1  and  as  stated  earlier,  the 
laser  camera  probe  allows  closely  representative  vertical  and  horizontal 
approach  distances  at  this  scale.  Modelboard  development  at  RSL  has 
shown  that  very  detailed  models  may  be  produced  at  1000:1  which  give  a 
variety  of  ground  texture  and  tree  detail,  these  should  enable  accurate 
height,  distance  and  speed  judgements  to  be  made. 
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In  addition,  pylons,  wires,  buildings,  and  vehicles,  have  been  modelled 
with  sufficient  realism  to  allow  examination  at  close  range,  eg  10-20 
mm  actual,  10-20  metres  visual. 

Conventional  camera  systems  are  unable  to  focus  on  such  close  objects 
without  blurring  the  more  distant  scene,  due  to  the  limited  'depth  of 
focus'  imposed  by  a  finite  entrance  pupil,  together  with  the  fact  that 
the  lens  system  operates  on  the  whole  field  of  view.  The  laser  has 
similar  limitations,  but  in  this  case  there  is  a  technique  available 
which  can  overcome  the  problem.  As  the  laser  beam  scans  across  the 
terrain  board  it  is  possible  to  re-focus  the  laser  spot  rcpidly  during 
the  line  scan  to  keep  all  objects  in  focus  and  hence  provide  a  large 
effective  depth  of  field. 

This  fast  focussing  device  incorporates  electro-optic  cells  that  are 
able  to  switch  the  plane  of  polarisation  of  the  laser  beam  which  when 
operated  in  conjunction  with  bi-refringent  lens,  has  the  effect  of 
selecting  one  of  the  two  focal  states  of  uhe  lens.  By  using  three 
cells,  eight  different  focal  states  are  provided  to  give  a  smooth 
change  of  focus  with  distance,  with  an  operating  speed  fast  enough 
to  optimise  the  resolution,  within  a  single  scan  line  for  objects 
from  5  mm  to  the  far  distance. 

A  range  finding  device,  which  would  probably  use  a  triangulation  method 
will  be  required  to  provide  the  focussing  device  with  the  appropriate 
switching  information. 

Light  gathering,  comparable  with  modelboard  illumination  in  a  conven¬ 
tional  camera/modelboard  system,  is  provided  by  photomultiplier  tubes 
and  will  be  produced  primarily  by  those  detectors  which  move  with  the 
camera.  These  will  be  arranged  in  a  series  of  concentric  circles 
around  the  probe  and  will  extend  for  a  radius  of  perhaps  two  metres 
scale,  corresponding  to  two  kilometres  visual  range.  If  more  distant 
scenes  are  required,  these  will  be  provided  by  fixed  photomultipliers 
arranged  in  much  the  same  way  as  the  fixed  lamp  bank  on  conventional 
systems . 

Pnotomultipliers  are  extremely  sensitive  to  light  and  will  produce  a 
video  signal  which  has  much  less  noise  content  than  a  typical  CCTV 
camera.  Also  the  electrical  power  required  to  drive  the  camera  lasers 
is  only  a  fraction  of  that  used  in  supplying  a  lampbank. 

The  combination  therefore  of  a  laser  camera,  photomultiplier  detectors, 
and  advanced  terrain  modelling  at  1000:1  scale,  can  produce  a  highly 
detailed,  in  focus,  wide-angle  image  that  should  provide  a  unique 
visual  capability  for  many  low-level  training  applications. 

Aircraft  that  may  require  this  type  of  image  generator  would  include 
attack,  scout  and  supply  helicopuers ,  anti-tank  and  ground  attack 
fixed  wing  aircraft  and  VTOL. 
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COMPUTER  GENERATED  IMAGERY 


The  projector  described  above  has  the  capacity  to  display  over  six  million 
picture  points  (for  comparison,  a  525  line  television  will  display  1/4 
million) . 

The  technology  of  CGI  has  now  advanced  to  the  stage  where  a  high  quality 
continuous  tone  image  extending  over  a  number  of  juxtaposed  conventional 
displays  can  easily  be  produced1*. 

The  scanned  laser  system  is  a  continuous  wide-angle  display.  The  180° 
by  60°  image  can,  however,  be  formed  by  segmenting  the  viewing  volume 
and  dedicating  a  single  CGI  channel  to  each  segment.  In  the  case  of 
the  Rediffusion/Evans  and  Sutherland  CT  5  image  generator,  it  would  be 
desirable  to  use  six  channels  and  to  increase  the  pixel  count  to  one 
million  per  channel.  Now  the  scanned  laser  system  scans  in  spherical 
polar  co-ordinates,  and  conventional  CGI  assumes  a  display  plane.  There 
is,  therefore,  a  need  to  either  convert  the  standard  image  to  spherical 
polar  form,  or  to  compute  the  image  directly  in  the  required  form.  The 
goals  set  for  the  design  of  a  CGI  system  compatible  with  the  scanned 
laser  system  were: 

a)  Preservation  of  scene  and  pixel  capacity  with  respect  to 
available  flat  plane  image  generation 

b)  Assure  the  straightness  of  edges 

c)  Assure  the  continuity  of  scenery  across  computational 
channel  boundaries 

d)  Assure  the  preservation  of  fog  and  intensity  gradients  to 
the  extent  that  they  reveal  or  suggest  shape  and  distance, 
and  that  Mach  banding  is  not  introduced 

e)  Assure  that  a  given  level  of  scene  quality  achieved  by  anti¬ 
aliasing  is  not  significantly  reduced 

f)  To  not  permit  any  increase  in  CGI  throughput  delay 

All  the  above  goals  can  be  met  by  redistributing  the  scan  on  the  image 
plane  so  that  spherical  pixels  are  generated  directly.  This  can  be 
achieved  with  little  impact  on  hardware  and  processing  capacity  by  a 
simple  change  of  parameters  in  the  nested  interpolation  formula  used 
to  convert  analytic  descriptions  of  scene  elements  into  imagery.  Such 
an  approach  preserves  the  in-built  anti-aliasing  measures  and  the 
continuity  of  imagery  between  computational  channels  without  adding  to 
the  throughput  delay. 

By  adopting  this  technique  it  is  possible  to  configure  a  CGI  system  in 
which  several  computational  channels  can  be  combined  to  produce  a  high 
quality  wide-angle  image  stored  in  a  frame  buffer.  This  store  can  then 
be  read  out  at  a  video  rate  compatible  with  the  scanned  laser  projector. 
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Computer  generated  imagery 

Wide-angle  displays  such  as  the  scanned  laser  projector  have  an  enormous 
appetite  for  pixels  if  the  images  are  projected  with  a  uniform  high 
resolution.  The  eye  acuity  curve  only  permits  the  viewer  to  see  a  high 
resolution  image  over  a  very  narrow  angle  about  the  fovea,  with  a  rapid 
fall  off  in  acuity  as  .the  eccentricity  is  increased.  An  image  generator 
which  produced  imagery  matched  to  the  resolution  capability  of  the  eye 
would,  in  principle,  use  the  minimum  amount  of  hardware  to  produce  the 
image.  In  practice,  the  production  of  eye  limited  resolution  image 
would  require  very  accurate  tracking  of  the  eye  pointing  direction  and 
an  image  generator  wifh  an  effective  zero  throughput  delay.  This  is 
unlikely  to  be  achieved  but  a  compromise  solution  in  which  a  high  resolu¬ 
tion  area  of  interest  is  moved  within  a  larger  field  of  view  with  lower 
resolution  imagery  is  possible  in  the  near  future. 

Display  devices 

There  is  a  trend,  for  air  combat  training,  for  example,  towards  large 
field  of  view  systems  with  resolution  as  good  as  the  unaided  eye.  The 
design  of  such  systems  poses  interesting  interactions  between  factors 
such  as  resolution,  system  bandwidth,  visibility  of  projectors,  image 
distortion,  brightness,  etc. 

As  discussed  in  the  previous  section,  the  system  bandwidth  can  best  be 
utilised  by  an  area  of  interest  type  display.  If  a  lightweight  projector 
is  mounted  on  the  pilot's  helmet,  then  it  will  move  with  his  head  and 
will  always  present  an  image  in  the  direction  the  pilot  is  looking.  It 
will  therefore  have  the  same  mobility  as  the  pilot's  head  and  will  enable 
a  training  simulator  to  be  built  in  which  the  total  field  of  view  will 
be  limited  only  by  the  cockpit  structure.  Such  a  projector  will  almost 
certainly  be  laser  based  to  give  the  required  resolution  and  will 
present  separate  right  and  left  eye  images  on  a  small  dome  screen  to 
give  the  correct  vergence  cues'*. 

SUMMARY  AND  CONCLUSIONS 


The  Rediffusion  scanned  laser  system  (Figure  3)  has  a  unique  potential 
for  displaying  continuous  wide-angle  scenes  from  either  a  modelboard 
or  CGI.  When  using  the  laser  camera  and  1000:1  modelboard  as  the  image 
generator,  a  very  detailed  image  will  be  produced  that  should  have 
particular  relevance  to  NOE  flying  training6. 

CGI  is  more  likely  to  be  used  for  ground  attack  or  air-to-air  combat 
situations.  The  visual  system  would  also  appear  to  be  suitable  for 
simulating  narrow  angle,  head  directed  vision  aids,  requiring  a  large 
field  of  regard. 
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Rediffusion  has,  through  several  separate  development  programmes  funded 
by  PM  Trade,  USAF,  MoD,  and  PVR  &  D,  proved  and  demonstrated  a  large 
part  of  the  technology  required  to  build  a  practical  visual  training 
aid. 
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INVESTIGATION  OF  SIMULATOR  DESIGN  FEATURES 
FOR  THE  CARRIER  LANDING  TASK 


ABSTRACT 

The  effects  of  eleven  factors  on  carrier  landing  performance  in  the 
Visual  Technology  Research  Simulator  (VTRS)  at  the  Naval  Training  Equip¬ 
ment  Center  (NTEC),  Orlando,  Florida,  were  investigated  within  a  single 
multifactor  experiment.  Subjects  for  the  experiment  were  eight  experienced 
naval  aviators.  Nine  display  and  simulator  factors  (Fresnel  Lens  Optical 
Landing  System  image,  ship  detail,  field  of  view,  visual  lags,  seascape, 
brightness,  TV  line  rate,  motion  and  engine  lags),  one  environmental  factor 
(turbulence)  and  subjects  as  a  factor  were  studied.  The  purpose  was  to  de¬ 
termine  and  rank  order  the  sizes  of  effects,  identify  factors  having  no 
effect,  and  to  obtain  information  for  making  decisions  about  future  transfer- 
of-training  studies.  Results  indicated  small  to  null  effects  for  equipment 
factors.  As  these  factors  were  manipulated  over  a  wide  range  of  interest 
representing  expensive  vs  inexpensive  simulation  options,  the  implication  is 
that  simulation  for  carrier  landing  skill  maintenance  and  transition  training 
does  not  require  the  more  costly  levels  of  fidelity  for  these  features. 
Simulator  requirements  for  training  at  the  undergraduate  level  are  currently 
being  examined. 

INTRODUCTION 

The  Visual  Technology  Research  Simulator  (VTRS)  at  the  Naval  Training 
Equipment  Center  (NTEC),  Orlando,  Florida,  has  been  designed  for  research  on 
flight  simulator  requi rements  for  training  and  skill  maintenance.  The  VTRS 
consists  of  a  fully  instrumented  Navy  T-2C  jet  trainer  cockpit,  a  six-degrees 
of-freedom  synergistic  motion  platform  and  a  wide  angle  visual  system  that 
can  project  both  computer-generated  and  model -board  images  onto  a  spherical 
screen.  The  visual  system  is  capable  of  displaying  images  via  target  and 
background  projectors  subtending  50°  above  and  30°  below  the  pilot's  eye 
level  and  can  display  160°  of  horizontal  field  (Collyer  and  Chambers,  1978). 

The  current  phase  of  effort  at  VTRS  involves  research  to  define  simu¬ 
lator  requirements  for  the  carrier  landing  task.  There  was  a  need  to  invest¬ 
igate  a  large  number  of  visual  and  other  simulator  features  of  interest  to 
simulator  designers  because  of  their  large  cost  implications.  It  was  con¬ 
sidered  necessary  to  search  for  a  more  economical  and  organized  method  of 
investigation  than  has  traditionally  been  employed  in  human  factors  research. 
A  research  program  was  therefore  planned  around  the  holistic  experimental 
philosophy  and  paradigm  discussed  by  Simon  (1977;  1979).  Fundamental  to 
this  approach  is  the  importance  of  studying  as  many  factors  of  interest  as 
possible  within  a  single  experiment.  In  so  doing,  the  information  obtained 
will  generally  be  more  accurate  and  comprehensive,  can  be  generalized  to 
more  operational  situations,  and  will  have  been  obtained  more  economically 
than  would  otherwise  be  possible.  This  holistic  approach  not  only  involves 
the  data  collection  plan,  but  also  covers  the  preparation,  analysis,  and 
interpretation  phases  of  a  program. 

The  experiment  reported  here,  which  investigated  the  performance  of 
experienced  pilots,  provided  information  regarding  the  absolute  and  relative 
effects  of  eleven  factors  and  their  critical  interactions.  It  also  provided 
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a  vehicle  for  exhaustively  testing  and  debugging  the  simulator  (.both  hardware 
and  software),  developing  new  performance  measures  and  refining  the  method¬ 
ology.  The  information  acquired  in  this  effort  is  directly  relevant  to  the 
design  of  simulators  for  skill  maintenance  and  transition  training,  and  is 
also  useful  for  the  study  of  requirements  for  undergraduate  training.  This 
experiment  culminated  the  first  stage  of  a  three-stage  program  that  is  an 
efficient  and  systematic  way  to  investigate  flight  simulator  design  features. 
In  such  a  program,  performance  studies  are  followed  by  "quasi -transfer" 
studies  (the  simulator  is  the  criterion  device  as  11  as  the  training  device) 
and  finally  by  actual  transfer  studies  (training  in  the  simulator  is  followed 
by  testing  in  the  airplane).  Information  acquired  at  each  stage  can  be  used 
to  make  decisions  regarding  research  strategy  and  selection  of  factors  to  be 
investigated  at  following  stages.  In  addition,  the  information  here  could  be 
reexamined  later  when  transfer-of-training  data  were  obtained  from  less 
experienced  pilots  in  order  to  see  the  relationships  that  exist  between  these 
two  bodies  of  data,  thereby  beginning  to  establish  general  principles 
relating  performance  to  transfer. 

METHOD 

Task 

The  task  was  to  land  the  simulated  T-2C  training  jet  on  the  aircraft 
carrier  Forrestal,  as  depicted  by  a  computer  image  generation  (CIG)  system. 

A  trial  consisted  of  a  straight-in  approach  flown  from  just  over  a  mile  from 
the  ship  with  the  aircraft  initialized  on  glideslope,  on  centerline,  and  at 
the  proper  angle  of  attack.  Although  a  circling  approach  is  part  of  the  day¬ 
time  carrier  landing  task,  the  decision  was  made  to  separate  it  from  the 
final  straight-in  segment.  The  turn  and  final  approach  are  in  fact  two  tasks 
separated  by  the  factors  likely  to  be  important,  the  nature  of  the  psycho¬ 
motor  skills  required  of  the  operator,  and  the  measurements  that  are  criti¬ 
cal.  Including  a  study  of  the  turn  with  the  final  approach  would  only  add 
variability  to  performance  which  would  mask  the  information  of  interest.  A 
separate  study  with  the  final  turn  was  done  for  factors  selected  as  poten¬ 
tially  having  an  effect  on  turn  performance  and  is  reported  in  Westra,  et  al. 
(1981).  - 

The  carrier  approach  and  landing  task  involves  a  family  of  visual  cues 
external  to  the  cockpit.  The  principal  cues  come  from  a  visual  landing  aid 
called  the  Fresnel  Lens  Optical  Landing  System  (FLOLS)  for  vertical  glide- 
slope  control;  carrier  deck  runway,  centerline  and  dropline  markings  for 
lineup  control;  and  the  sky,  horizon,  and  seascape  for  general  aircraft  atti¬ 
tude  control. 

Factors 

Many  factors  originally  designated  as  candidates  for  study  were  pared 
down  by  a  panel  of  engineers  and  psychologists  into  the  set  of  ten  factors 
shown  in  Table  l.1  This  final  list  of  factors  represented  a  number  of  design 


‘Two  other  simulator  factors  with  large  cost  implications  which  were  part 
of  the  first  research  effort  but  not  included  in  the  experiment  presented  here 
were  G-seat  (on  or  off)  and  ship  type  (computer-generated  image  or  model - 
board).  Results  for  these  two  factors  are  reported  in  Westra,  et  al.  (1981). 


TABLE  1.  FACTORS  AND  SUMMARY  OF  LEVELS  SELECTED 


FACTOR 

LEVEL  SETTINGS 

"low" 

"high" 

FLOLS 

TV/CIG 

Opti cal /Model 

Ship  Detail 

Night  point-light 

Day  Solid  Surface 

Field  of  View 

-27°  +  9°  Vertical 

-30°  to  50°  Vertical 

+  24°  Horizontal 

+  80°  Horizontal 

Visual  Lags 

200  msec  Total  Delay 

100  msec  Total  Delay 

Seascape 

Gray  Homogeneous 
Background 

Wave  Pattern 

Brightness 

Ship:  0.40  fL 

2.80  fL 

Sea:  0.04  fL 

0.50  fL 

Sky:  0.02  fL 

0.16  fL 

TV  Line  Rate 

525 

1025 

Motion 

Fixed  Base 

S i x- Degrees -of - F  reedom 

Engine  Lags 

7*5  Hz  Update 

30  Hz  Update 

Turbulence 

Close  to  Maximum 
Flyable 

No  Turbulence 

considerations  relevant  to  carrier  landing  trainers*  They  were  considered  to 
be  the  most  important  in  terms  of  cost  implications  and  potential  effects. 
These  factors  could  generally  be  categorized  as  display  and  simulator  hard¬ 
ware  fidelity  variables.  Seven  of  these  factors  were  visual  system  parameters 
which  directly  affected  the  quality  of  the  visual  cues.  Training  aids  or 
augmented  simulation  features  were  not  considered  for  this  performance  study 
as  they  cannot  be  meaningfully  studied  apart  from  their  effect  on  learning 
and  transfer  of  training. 

The  goal  in  choosing  the  factor  level  settings  was  to  bracket  the 
reasonable  range  of  interest.  Thus  "high"  and  "low"  levels  for  each  factor 
were  set  with  this  in  mind.  Most  factors  had  their  high  levels  set  at  the 
"best"  or  optimum  engineering  levels  attainable  while  the  low  levels  were 
chosen  to  be  the  most  degraded  form  of  the  factor  still  within  the  reasonable 
range  of  interest.  Some  factor  level  settings  cannot  necessarily  be  placed 
in  the  above  categories  and  were  simply  comparisons  of  two  design  options 
differing  in  cost. 

The  FLOLS  high  level  was  an  optical  projection  of  a  scale  model  of  the 
FLOLS  providing  essentially  real-world  duplication  of  the  24  ft.  wide  array 
of  lights  except  for  a  constant  magnification  of  1.5X.  This  high  resolution 
direct  optical  projection  was  specially  built  for  this  test  to  represent  maxi¬ 
mum  realism.  It  is  not  in  current  trainers  and  would  be  a  high  cost  addi¬ 
tional  subsystem.  The  FLOLS  low  level  was  a  TV  projection  of  a  CIG  FLOLS 
data  base  defined  by  96  edges.  The  two  datum  bars  (stationary  elements  used 
as  reference  lines  for  the  moving  element  known  as  the  "meatball")  which  nor¬ 
mally  consist  of  six  lights  each  were  represented  by  two  solid  bars.  At  long 
range  the  FLOLS  was  magnified  three  times  normal  size,  gradually  shrinking 
to  1.5X  at  the  ramp.  The  large  magnification  was  required  to  compensate  for 
limited  TV  resolution  so  that  the  pilots  could  discriminate  meatball  position 
for  gli deslope  guidance  at  a  range  similar  to  the  real  world.  This  FLOLS 
magnification  technique  is  regularly  used  on  Navy  carrier  landing  trainers 
with  the  TA-4J  trainer  (device  2B35)  using  magnifications  as  large  as  7X. 

This  technique  involves  no  added  cost  to  CIG  carrier  landing  visual 
equipment. 

The  ship  detail  high  level  was  a  daytime  solid  model  CIG  carrier  whose 
surfaces  were  defined  by  985  edges.  It  contained  all  landing  deck  markings 
and  was  scaled  normal  size.  This  detail  level  was  representative  of 
that  available  from  daytime  CIG  systems  costing  several  million  dollars  such 
as  the  2B35  trainer,  although  displayed  at  higher  resolution  than  available 
in  the  2B35.  The  ship  detail  low  level  was  a  night  point-light  CIG  carrier 
deck  consisting  of  137  lights.  It  contained  all  deck  outline,  runway,  center- 
line  and  drop  lights,  with  the  runway  surface  becoming  visible  during  the 
final  1600  ft.  of  the  approach.  Its  image  is  representative  of  a  night  CIG 
system  costing  less  than  a  million  dollars  which  is  used  on  several  Navy 
Night  Carrier  Landing  Trainers  (NCLT's). 

The  field  of  view  high  level  was  a  160°  H  by  80°  V  wide  angle  display 
which  is  costly  and  is  representative  of  that  currently  available  for 
carrier  landing  training  only  on  multitask  trainers  such  as  the  2B35  and  the 
F- 14  Wide  Angle  Visual  System  (WAVS).  The  field  of  view  low  level  was  a  48°  H 
by  36°  V  narrow  angle  display  which  is  representative  of  the  lower  cost  NCLT's 
used  for  F-4,  F-14,  A-6,  A-7,  and  S-3  transition  training. 
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The  visual  lag  high  level  of  100  msec  from  stick  input  to  the  completion 
of  the  first  field  of  video  output  is  representative  of  a  3Q  Hz  update  com¬ 
puter  simulation  and  a  60  Hz  CIG  update.  This  response  time  is  faster  than 
available  on  current  Navy  trainers  and  represents  increased  computer  capacity 
and  cost.  The  visual  lag  low  level  of  200  msec  is  representative  of  a  15  Hz 
update  computer  simulation  and  a  30  Hz  CIG  update.  This  response  time  is 
representative  of  the  lowest  level  normally  considered  in  simulator  acquisi¬ 
tions  with  visual  systems. 

The  seascape  high  level  was  a  water  texture  pattern  generated  by  a 
flying  spot  scanner  readout  of  a  photographic  film  plate  of  a  seascape  wave 
pattern.  It  provided  translation  and  ground  growth  as  well  as  attitude 
(i.e.,  roll,  pitch,  yaw)  information  considered  to  be  potentially  useful  as 
cues  for  improved  aircraft  control  and  visual  realism.  The  seascape  low 
level  was  the  absence  of  the  texture  pattern  and  the  use  of  a  uniform  grey 
level  below  the  horizon,  providing  roll  and  pitch  information  only.  This, 
of  course,  represents  a  reduced  cost  by  eliminating  the  seascape  image 
generator. 

The  brightness  high  level  of  2.8  foot-Lamberts  (fL)  for  the  target  is 
representative  of  most  narrow  angle  visual  systems  used  on  trainers.  The 
brightness  low  level  of  0.4  fL  is  representative  of  most  wide  angle  dome 
displays  such  as  the  F-14  WAVS.  To  increase  the  wide  angle  display  bright¬ 
ness  to  the  higher  level,  if  found  advantageous  for  carrier  landing  training, 
would  be  very  expensive  and  would  require  considerable  development  time. 

The  TV  line  rate  high  level  of  1025  is  the  maximum  available  in  standard 
TV  equipment.  The  TV  line  rate  lower  level  of  525  provides  only  half  the 
resolution  but  represents  low  cost  commercial  equipment.  This  factor  as 
used  in  this  experiment  relates  only  to  narrow  angle  projectors  which  use 
zoom  magnification  to  provide  high  resolution  displays  such  as  in  the  F-14 
WAVS. 


Three  additional  factors  not  involving  visual  equipment  were  included 
in  the  study.  A  si x-degrees-of- freedom  48  inch  synergistic  motion  platform 
was  fully  operational  for  the  high  level  and  stationary  for  the  low  level. 
This  platform  is  similar  to  those  on  the  Navy's  27  T-2C  Instrument  Trainers 
(device  2F101)  used  in  Undergraduate  Pilot  Training  (UPT)  except  that  VTRS 
computation  rates  are  higher  for  reduced  cuing  time  lag.  While  it  is  repre¬ 
sentative  of  many  older  platforms  on  existing  trainers,  it  does  not  have  the 
low  noise  and  improved  response  of  new  platforms. 

Engine  computation  time  lags  were  set  at  30  Hz  and  7.5  Hz  for  high  and 
low  levels.  This  factor  was  included  because  the  rather  large  engine  compu¬ 
tations  are  typically  done  at  low  rates  such  as  7.5  Hz  to  minimize  computer 
size  and  cost.  Preliminary  testing  had  generated  positive  reactions  by 
pilots  that  the  30  Hz  engine  updates  caused  the  simulator  response  to  feel 
more  realistic.  This  was  considered  important  since  Navy  carrier  landing 
approach  procedures  for  jet  aircraft  typically  require  frequent  throttle 
changes  for  glideslope  control. 

Turbulence  was  included  in  the  experiment  to  allow  examination  of  factor 
effects  under  two  difficulty  levels.  The  high  level  was  set  at  no  environ¬ 
mental  turbulence  (in  this  case  the  no  turbulence  level  was  called  "high" 
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as  it  was  presided  to  result  in  better  performance).  The  low  level  setting 
was  turbulence  in  the  form  of  wind  acting  on  the  longitudinal,  lateral  and 
vertical  aircraft  axes.  These  "winds"  were  generated  by  the  sunination  of 
sine  waves  varying  in  frequencies  and  amplitudes.  They  were  strong  enough  to 
create  a  degree  of  turbulence  judged  by  Navy  pilots  to  be  near  the  maximum 
level  under  which  operations  would  proceed  at  sea. 

Subjects 

The  eight  pilots  were  experienced  naval  aviators  with  at  least  one  tour 
at  sea.  The  average  number  of  military  flight  hours  for  these  pilots  was 
2254,  ranging  from  630  to  4500.  The  average  number  of  carrier  landings  (non¬ 
simulator)  was  346,  ranging  from  a  low  of  50  to  a  high  of  800.  Five  of  the 
pilots  had  most  recent  flight  experience  in  the  A-7  aircraft,  two  in  the  S-3 
and  one  in  the  A-4.  All  were  volunteers  stationed  at  Naval  Air  Station  Cecil 
Field,  Florida,  at  the  time  of  the  research. 

Design 

The  basic  design  was  one-sixteenth  of  the  fully  crossed  full  factorial 
211  and  required  128  experimental  conditions  separated  into  four  blocks. 

This  design  permitted  each  main  effect  and  each  two-factor  interaction  to  be 
isolated  from  one  another.  Main  effects  were  confounded  with  some  three- 
factor  interactions,  but  the  effect  of  some  other  three-factor  interactions 
(in  strings)  could  be  estimated.  The  design  was  modified  by  dividing  the 
conditions  within  each  original  block  into  two  blocks  of  16  conditions  each 
in  order  to  test  a  different  pilot  on  each  block,  two  trials  per  condition. 

No  main  effects  and  only  one  two-factor  interaction  were  confounded  with 
blocks,  i.e.,  mean  differences  among  the  pilots.2 

The  replication  employed  in  the  design  as  well  as  its  resolution  and 
size  is  normally  not  in  keeping  with  the  multi  factor  strategy  of  economy  at 
this  stage  of  a  research  program.  No  new  information  is  obtained  regarding 
factors  by  replicating  and  more  isolation  of  higher-order  effects  would  have 
been  obtained  had  a  different  fraction  of  the  total  been  employed.  However, 
preliminary  investigations  pointed  out  several  problems  that  showed  the  need 
for  replication  and  fairly  large  sample  size.  First,  observed  trial-to-trial 
reliability  was  low  and  examination  of  two  trials  both  separately  and  com¬ 
bined  would  provide  better  understanding  of  the  data.  Second,  it  became 
apparent  that  many  of  the  factors  studied  were  having  little  or  no  meaningful 
effect  on  performance.  As  a  number  of  these  had  large  cost  implications,  it 
was  considered  important  to  be  able  to  accept  the  null  hypothesis  with  con¬ 
fidence.  The  large  sample  size  was  needed  to  decrease  the  probability  of 
being  wrong  when  stating  that  certain  factors  had  no  practical  effects. 

Performance  Measures 

A  great  many  performance  measures  were  examined.  These  measures  included 
outcome  scores  defined  in  terms  of  the  task  itself  such  as  glideslope,  lineup. 


2The  interaction  of  engine  lags  with  brightness  was  chosen  to  be  con¬ 
founded  with  blocks  as  in  the  authors'  opinion  it  has  one  of  the  lowest 
probabilities  of  being  important. 
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and  angle  of  attack  deviations  for  various  approach  segments  as  well  as  touch¬ 
down  performance.  Intermediate  aircraft  measures  of  roll  and  pitch  control 
and  input  measures  of  stick  activity  were  also  taken.  Deviations  for  glide- 
slope,  angle  of  attack,  lineup,  roll,  and  pitch  were  summarized  over  approach 
segments  with  root  mean  square  (RMS)  scores  which  were  further  broken  down 

into  the  independent  bias  and  variability  components  of  RMS.  This  follows 

_  2  2 

from  the  relationship  Zet2/n  =  e  +  l(et-e)  /n  where  et  is  error  at  time  t,  n  is 

the  number  of  samples  taken  and  the  square  root  of  the  term  on  the  left  is 
RMS  error.  Bias  and  variability  components  were  subjected  to  square  root 
transformations.  Deviations  were  measured  both  in  feet  and  degrees  for  glide- 
slope  and  lineup.  Stick  summary  measures  for  aileron,  elevator,  throttle, 
and  pedal  were  taken  simply  as  total  stick  movement  (unsealed)  over  a  segment. 
The  segments  were  broken  down  roughly  into  distances  representing  "in  the 
middle",  "in  close",  and  "at  the  ramp"  (aft  end  of  landing  deck)  which  the 
Navy  has  traditionally  used  to  define  segments.  These  distances  were  4000' 
to  2000'  from  the  ramp,  2000'  to  500'  from  the  ramp,  and  1000*  to  the  ramp, 
respectively. 

A  number  of  quality  scores  were  created  in  an  attempt  to  provide  infor¬ 
mation  that  could  be  interpreted  directly  in  terms  of  "goodness"  of  perfor¬ 
mance,  that  is,  measures  that  are  clearly  related  to  how  accurately  and 
safely  the  aircraft  arrives  at  the  landing  deck.  Quality  of  landing  scores 
were  created  for  touchdown  lineup,  roll,  pitch,  vertical  velocity,  and  wire 
trapped.  These  scores  were  scaled  from  zero  to  100  with  zero  indicating  per¬ 
formance  that  was  at  the  point  of  causing  damage  to  the  aircraft  or  was 
totally  outside  of  acceptable  limits.  A  score  of  100  indicated  performance 
within  a  fairly  narrow  range  defining  "perfect"  touchdown.  The  wire  trapped 
score  was  based  partially  on  Brictson's  (1973)  Landing  Performance  Score 
(LPS) .  Additionally,  percent  Time-on-Target  (TOT)  Scores  were  created  for 
glideslope,  lineup,  and  angle  of  attack  for  the  "at  the  ramp"  segment  of 
1000*  to  the  ramp.  Based  on  recommendations  by  Navy  Landing  Signal  Officers 
(LSO's),  tolerance  bands  for  on-target  glideslope,  lineup,  and  angle  of 
attack  performance  were  +  .3°,  +  1.5°,  and  +  .5  unit,  respectively. 

RESULTS  AND  DISCUSSION 

Both  univariate  and  multivariate  analyses  were  performed.  Since  space 
limits  the  amount  that  can  be  presented  here,  only  some  of  the  more  important 
and  interesting  findings  will  be  discussed.  The  interested  reader  is  urged 
to  consult  Westra,  et  al .  (1981)  for  a  detailed  presentation  of  results. 
Univariate  results  wilT~be  presented  for  two  measures,  RMS  vertical  glide- 
slope  error  and  roll  variability,  both  in  the  segment  between  2000*  to  5C0' 
from  the  ramp.  Multivariate  results  will  be  shown  for  seventeen  measures 
which  are  directly  interpretable  in  terms  of  the  quality  of  performance. 

The  columns  in  Tables  2  and  3  are  defined  as  follows.  "Effect"  is  the 
difference  obtained  when  subtracting  the  mean  of  the  low  level  condition 
from  the  mean  of  the  high  level  condition.  "Percent  total  variance"  indi¬ 
cates  the  relative  contribution  each  source  made  to  the  total  variability 
of  performance  within  the  experiment.  It  is  obtained  by  dividing  the  sum 
of  squares  for  that  source  of  variance  by  the  total  sum  of  squares.  "Mean 
square"  is  the  variance,  or  the  sum  of  squares  per  degree  of  freedom  for  the 
particular  source.  "F"  is  the  ratio  of  the  specific  variance  for  a  source 
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over  the  residual  variance.  The  residual  variance  is  actually  a  composite  of 
all  effects  of  trials  as  well  as  four-factor  and  higher  interactions  that 
could  be  isolated.  (The  trials  effect  was  examined  initially  for  the  presence 
of  systematic  differences  and  none  was  found  to  exist.)  It  was  used  as  is 
conventionally  done  as  the  error  term. 

RMS  Glides lope  Deviations 

Table  2  shows  the  results  of  the  analysis  of  the  RMS  vertical  glideslope 
deviations  in  degrees  in  the  flight  segment  between  2Q00'  and  500'  from  the 
ramp.  The  analysis  of  variance  reveals  that  turbulence,  subjects,  and  the 
FLOLS  are  the  only  sources  of  variance  with  effects  larger  than  one  might 
expect  by  chance.  The  analysis  also  shows  that  while  the  cumulative  effects 
of  two-  and  three-factor  interactions  account  for  a  good  deal  of  the  variance 
in  this  experiment,  this  variability  had  a  high  probability  of  occurring 
purely  by  chance.  Furthermore,  with  the  large  number  of  interactions  involved 
(44  two-factor  and  59  three-factor)  on  average  the  contribution  of  each  is 
small,  being  0.23  and  0.49  percent  of  total  variance,  respectively.  Two- 
factor  interactions  were  examined  separately  and  summed  into  a  single  term 
only  after  determining  that  none  was  large  enough  to  be  considered 
important. 

Neither  the  large  turbulence  effect  nor  the  subject  effects  come  as  a 
surprise  since  turbulence  was  designed  into  the  experiment  to  provide  at 
least  two  levels  of  task  difficulty  and  individuals  ostensively  from  the 
same  population  do  differ.  The  experimental  design  was  constructed  so  that 
differences  among  each  subject's  average  performance  would  not  be  confounded 
with  any  of  the  main  effects  or  interactions  of  interest. 

Of  the  equipment  factors,  only  the  FLOLS  effect  stood  out  from  the  rest 
for  this  measure.  The  magnitude  of  the  FLOLS  effect  is  .075°  of  RMS  glide- 
slope  error  which  is  approximately  1/4  of  the  angle  subtended  by  one  of  the 
five  separate  columnar  FLOLS  lenses.  The  sign  of  the  effect  indicates  that 
the  "low"  condition  which  was  the  CIG  FLOLS  had  a  lower  mean  RMS  glideslope 
error  than  the  optical  FLOLS.  This  result  was  surprising  since  the  optical 
FLOLS  was  considered  the  "better"  higher  fidelity  display.  However,  the 
CIG  FLOLS  had  been  increased  in  size  to  compensate  for  its  poorer  resolution 
such  that  it  would  be  visible  from  approximately  the  same  range  as  the  opti¬ 
cal  FLOLS.  The  CIG  FLOLS  was  then  gradually  reduced  in  size  until  it  was  the 
same  relative  size  as  the  optical  FLOLS  at  the  ramp. 

Roll  Variability 


Table  3  shows  results  for  the  measure  of  roll  variability  in  degrees 
from  2000'  to  500'  from  the  ramp.  This  is  a  more  sensitive  and  less  global 
measure  of  an  intermediate  control  loop  and  has  a  smaller  proportion  of 
"error"  variance  associated  with  it.  Considering  the  large  sample  size 
involved,  the  relative  power  for  this  measure  is  very  high.  Examination  of 
the  percent  variance  column  shows  field  of  view  and  visual  lags  had  the 
largest  effects  of  the  equipment  variables  accounting  for  4.27%  and  3.51%  of 
the  variance  in  the  experiment,  respectively.  The  narrow  field  of  view 
resulted  in  greater  roll  variability  than  the  wide  field  of  view.  Results 
agree  with  expectations  here  since  it  was  felt  that  the  wide  horizon  on  a 
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TA8LE  3.  RESULTS  FOR  ROLL  VARIABILITY  IN  DEGREES 
2000'  TO  500'  FROM  THE  RAMP 
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confounded  with  lower  order  terms. 


straight-in  approach  could  be  useful  to  the  pilot  as  peripheral  information 
to  aid  in  keeping  wings  level.  Preliminary  investigation  also  had  indicated 
that  increased  visual  lags  would  cause  increased  roll  variability  (until  the 
delay  is  so  long  that  the  pilot  stops  trying  to  make  precise  corrections)  and 
results  confirm  this.  Whether  these  effects  are  of  practical  importance  is 
another  issue  which  must  take  into  account  their  effect  on  the  outcome  of  the 
task  among  other  things. 

Brightness  shows  up  as  a  statistically  significant  effect  in  Table  3 
but  it  only  accounts  for  1.3%  of  the  variance  and  the  magnitude  seems  small 
(although  there  is  no  good  barometer  for  determining  "small"  or  "large"  for 
this  measure).  Thus  this  effect  was  judged  real  but  relatively  minor.  Roll 
variability  is  sensitive  to  turbulence  effects  and  also  to  subject  differences 
as  noted  by  the  large  amounts  of  variance  accounted  for  by  these  factors.  The 
size  of  subject  effects  is  an  item  that  an  investigator  wishing  to  interpret 
results  should  not  ignore.  When  other  effects  are  small  relative  to  subject 
effects,  even  though  statistically  significant,  general izability  is  restricted 
until  the  causes  of  subject  variability  are  better  understood. 

Quality  of  Performance 

An  attempt  was  made  to  provide  some  information  for  interpreting  results 
in  a  general  or  overall  sense  in  terms  of  the  quality  of  the  approach  and 
landing.  Multivariate  analyses  were  performed  for  seventeen  quality  indices 
including:  RMS  glides lope,  lineup,  and  angle-of-attack  error  for  segments 
6000'  to  4000',  4000'  to  2000',  and  2000'  to  500'  from  the  ramp;  percent 
time  on  target  for  glideslope,  lineup,  and  angle  of  attack  for  1000’  to  the 
ramp;  and  the  touchdown  performance  scores  described  earlier  for  touchdown 
lineup,  roll,  pitch,  vertical  velocity,  and  wire  trapped.  One-factor  discrim¬ 
inant  analyses  were  performed  for  each  factor  to  provide  a  multivariate 
ranking  of  the  magnitude  of  effects.  In  each  discriminant  analysis,  the 
levels  of  the  factor  were  designated  as  "groups"  and  the  discriminant  proce¬ 
dure  finds  the  linear  combination  of  the  dependent  measures  which  maximally 
discriminates  between  groups.  The  relative  degree  to  which  groups  can  be 
discriminated  then  provides  for  a  multivariate  ranking  of  the  factors. 

Table  4  shows  the  results  of  the  discriminant  analyses.  The  factors  are 
listed  in  the  order  of  greatest  differences  between  levels  of  factors. 

Ranking  was  based  on  the  percent  of  cases  correctly  classified  into  groups 
based  on  the  discriminant  function.  A  jackknifed  procedure  was  used  to  pre¬ 
dict  group  membership  which  helps  reduce  bias  by  making  a  prediction  for  a 
trial  based  on  all  information  except  information  from  that  trial.  Note 
that  a  two-group  correct  classification  of  50%  would  be  expected  by  chance 
alone.  The  multiple  R  is  the  multiple  correlation  between  the  dependent 
variables  and  the  levels  of  the  factor  and  the  multivariate  F  is  an  approxi¬ 
mation  based  on  Wilks'  Lambda.  (The  ranks  based  on  R  and  F  do  not  correspond 
exactly  to  the  rank  based  on  percent  classification  because  the  jackknifed 
procedure  removes  some  of  the  bias  inherent  in  multiple  R  with  correlated 
measures. ) 

As  Table  4  shows,  of  the  equipment  factors,  only  the  FLOLS  has  an 
overall  effect  which  is  clearly  greater  than  chance  as  indicated  by  the 
multivariate  F.  Most  of  the  effect  was  in  glideslope  control  whicn  was 
better  for  the  CIG  FLOLS.  Lineup  control  was  also  better  with  the  CIG  FLOLS 


459 


TABLE  4.  DISCRIMINANT  ANALYSES  RESULTS 
USING  SEVENTEEN  QUALITY  INDICES 


JACKKNIFED  % 

FACTOR  CORRECT  CLASSIFICATION 

MULTIPLE  R 

MULTIVARIATE  F1 

FLOLS 

61.4 

.41 

2.80* 

Ship  Detail 

57.0 

.32 

1.44 

Seascape 

51.4 

00 

1.12 

Line  Rate 

51.0 

.26 

0.99 

Motion 

49.4 

.26 

1.12 

Engine  Lags 

48.2 

.28 

1.15 

Field  of  View 

47.0 

.22 

0.71 

Brightness 

45.0 

.22 

0.71 

Visual  Lags 

44.6 

.22 

0.67 

Turbulence 

68.5 

.50 

4.67* 

Subjects2 

40.6 

.66 

4.13* 

‘Approximate  F  based  on  Wilks'  Lambda  with  17  and  233  degrees  of  freedom. 

2First  discriminant  function  only.  Since  there  were  8  pilots,  the  expected 
percent  correct  classification  is  12.5.  F  is  based  on  119  and  1490 
degrees  of  freedom. 

*Probability  £  .05 
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in  the  middle  of  the  approach.  Touchdown  performance  was  not  affected  by 
FLOLS  type  to  an  appreciable  extent.  Ship  detail  ranks  next  in  terms  of 
overall  effect  with  57%  correct  classification.  This  is  slightly  better  than 
chance  and  although  the  multivariate  F  is  not  significant  it  must  be  kept  in 
mind  that  all  other  sources  of  variance  are  in  the  "error"  term  including  tur¬ 
bulence  and  pilots.  Thus  ship  detail  was  judged  as  having  a  real  effect  on 
the  quality  of  performance.  The  effect  was  most  noticeable  in  lineup  control 
which  was  better  with  the  highly  detailed  ship  in  the  middle  and  in  close. 
Angle-of-attack  control  at  the  ramp  and  the  wire-trapped  score  were  moderately 
better  with  the  high  detail  ship.  No  other  equipment  factor  appears  to  be 
operating  at  more  than  chance  in  terms  of  overall  quality  of  the  task. 

SUMMARY  AND  CONCLUSIONS 

The  results  for  the  measures  presented  here  as  well  as  for  many  other 
measures  examined  reveal  that,  generally,  equipment  effects  were  small  rela¬ 
tive  to  subject  differences.  Since  the  pilots  were  all  experienced  naval 
aviators  with  at  least  one  tour  at  sea  and  their  differences  in  performance 
still  overshadowed  equipment  differences,  there  is  an  implication  that  a 
point  of  diminishing  returns  with  regard  to  further  simulator  fidelity 
improvements  has  been  reached  for  this  task.  Only  one  of  the  nine  equipment 
factors  studied  was  judged  to  have  meaningfully  affected  the  quality  of  per¬ 
formance  and  one  other  was  judged  to  have  marginally  affected  the  quality 
of  performance.  Seven  other  equipment  factors  were  shown  to  have  no  effect 
on  the  quality  of  performance  although  some  of  these  did  have  specific  real 
effects  on  some  dimensions  of  performance.  A  brief  summary  of  these  effects 
by  factor  follows. 

FLOLS  -  The  CIG  FLOLS  resulted  in  improved  gli deslope  control  compared  to  the 
optical  FLOLS.  Touchdown  performance  was  not  affected. 

Ship  Detail  -  The  night  point-light  ship  seemed  to  cause  a  lineup  bias  to  the 
left.  This  bias  decreased  in  close  to  the  ramp  although  touchdown  performance 
for  both  lineup  and  wire  trapped  was  slightly  worse  than  with  the  fully 
detailed  CIG  ship. 

Field  of  View  -  The  narrow  field  of  view  increased  roll  variability  throughout 
the  approach  relative  to  the  wide  field  of  view.  Although  this  was  a  real 
effect  and  may  suggest  increased  workload,  no  effect  on  the  quality  of  per¬ 
formance  was  noted. 

Visual  Lags  -  The  longer  visual  delay  also  increased  roll  variability  through- 
out  the  approach.  No  effect  on  the  quality  of  performance  was  noted. 

Motion  -  Pilot  and  aircraft  activity  in  general  seemed  to  be  increased  with 
the  motion  on,  particularly  elevator  input  and  pitch  variability.  This 
increased  activity  did  not  meaningfully  affect  the  overall  quality  of  per¬ 
formance.  However,  analyses  of  some  specific  measures  suggested  an  inter¬ 
action  of  motion  with  turbulence.  With  turbulence,  the  motion  system 
improved  the  touchdown  lineup  and  wire-trapped  performance  moderately  while 
with  no  turbulence,  motion  had  no  effect  or  a  slightly  deleterious  effect. 

Engine  Lags,  Seascape,  Brightness,  and  Image  Line  Rate  -  Although  in  a  few 
cases  there  were  specific  effects  which  were  statistically  significant, 
generally  no  more  than  1%  of  the  variance  was  accounted  for  in  any  measure 
by  any  of  these  factors.  There  was  no  evidence  that  they  had  a  meaningful 
effect  on  performance. 
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As  factors  were  manipulated  over  a  wide  range  of  interest  generally 
representing  expensive  vs  inexpensive  alternatives,  the  implication  is  that 
simulation  for  carrier  landing  skill  maintenance  and  transition  training 
does  not  require  the  greater  fidelity  represented  by  the  high  levels  tested 
in  this  experiment.  The  lower  levels  tested  resulted  in  little  or  no  perfor¬ 
mance  decrement,  and  represent  relatively  low  cost  options  for  dedicated 
part-task  carrier  landing  trainers  such  as  NCLT*s.  Multitask  trainers  pro¬ 
viding  higher  levels  of  fidelity  (which  may  be  necessary  for  other  tasks) 
would  not  be  expected  to  provide  better  carrier  landing  skill  maintenance 
or  transition  training. 

« 

The  implications  for  training  at  the  UPT  level  will  be  verified  in 
research  now  underway.  Results  of  this  experiment  have  been  used  to  make 
decisions  about  the  next  phases  of  the  carrier  landing  program,  including  the 
selection  of  factors  to  be  carried  forward  into  transfer-of-training  research. 
Experiments  using  quasi-transfer  and  actual  transfer-of-training  paradigms 
with  inexperienced  undergraduate  pilots  will  determine  the  effects  of  ship 
detail,  field  of  view,  FLOLS  size,  motion,  and  other  factors. 

This  multi  factor  approach  provided  much  information  not  available  from 
typical  few- factor  experiments.  In  particular,  the  overview  that  was 
obtained  put  the  relative  importance  of  the  factors  into  a  better  perspective. 
Results  are  more  general izable  because  of  the  many  conditions  under  which 
performance  was  investigated,  all  of  which  were  of  interest  and  concern. 

The  existence  of  two-factor  interactions  has  been  examined  far  less  expen¬ 
sively  and  more  completely  than  had  any  subset  of  factors  been  studied  a 
few  at  a  time.  The  approach  is  recommended  to  others  concerned  with  iden¬ 
tifying  the  necessary  and  sufficient  characteristics  of  training  devices. 
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SYSTEM  STRATEGIES  TO  OPTIMIZE 
CIG  IMAGE  CONTEST 

M.  COSMAN  and  R.  SCHUMACKER 
Evans  and  Sutherland  Computer  Corporation 

ABSTRACT 

Along  with  the  rapidly  increasing  capability  of  computer  image  genera¬ 
tion  equipment  has  come  an  increasing  appreciation  of  the  role  of  database 
design,  structure,  and  management  in  providing  effective  training  imagery. 
This  paper  describes  the  techniques  used  in  CT-5  image  generation  systems  to 
organize  and  manage  the  database,  and  to  efficiently  focus  the  system  resources 
on  producing  images  of  maximum  complexity  and  quality.  Approaches  to  database 
design  that  exploit  these  system  characteristics  are  discussed  and  illustrated 
by  photographs  taken  from  production  equipment. 

INTROOJCTICN 

As  computer  image  generation  technology  moves  into  an  era  of  intense  and 
accelerated  development,  the  requirements  placed  on  such  training  equipment 
seem  always  to  outpace  the  available  capability.  The  quest  for  systems  and 
techniques  which  provide  a  meaningful  and  realistic  training  environment  is 
intensifying,  and  increased  emphasis  is  being  given  to  the  issues  of  image 
quality  and  image  usefulness.  The  CT-5  systems  now  in  production  at  Evans  & 
Sutherland  address  these  basic  issues  by  incorporating  a  comprehensive  solution 
to  the  problem  of  image  quality,  while  implementing  an  approach  which  provides 
dramatic  improvements  in  routinely  achievable  image  complexity.  Special  atten¬ 
tion  is  given  to  the  problem  of  managing  extremely  large  databases,  while 
making  sure  that  the  image  generation  hardware  spends  a  maximum  amount  of  its 
resources  processing  scene  details  which  are  relevant  to  the  training  mission. 

Several  basic  philosophies  and  concepts  guided  the  development  of  the 
CT-5  architecture.  Cne  was  the  belief  that  simulated  imagery  should  be  as  free 
as  possible  from  any  distracting  visual  artifacts  which  reduce  the  impression 
of  realism,  or,  worse  yet,  constitute  negative  training  cues.  Thus  image 
quality,  particularly  lack  of  any  of  the  symptoms  of  aliasing  so  characteristic 
of  sane  previous  systems,  was  a  prime  design  goal.  Another  was  the  notion  that 
system  capacity  must  be  more  cleverly  used  in  order  to  be  maximally  effective, 
hence  a  database  development  and  management  strategy  which  emphasizes  providing 
scene  details  of  greatest  visual  significance. 

This  paper  will  discuss  three  main  areas  of  the  CT-5  technology  in  some 
detail:  The  cxsplay  processor  and  its  impact  on  image  quality;  the  viewpoint 
processor  and  its  role  in  providing  relevant  scene  details;  and  an  approach  to 
database  structure,  design  and  development  which  effectively  supports  and 
exploits  the  capabilities  of  the  first  two. 

SYSTEM  OVERVIEW 

Figure  1  shows  a  basic  configuration  of  CT-5  components  that  provides  a 
single  channel  capability  at  standard  (525-line)  resolution.  The  major 
sections  include  a  general  purpose  computer  (a  Digital  Equipment  Corporation 
PDP-11 ) ,  special  purpose  hardware  called  an  image  processor,  and  a  display. 
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EXPANSION 


Figure  1  Basic  CT-5  Configuration 


The  general  purpose  computer  interfaces  to  the  image  processor  and  to 
the  host  computer,  and  monitors  and  manages  the  visual  system,  it  hosts  the 
magnetic  disk  on  which  the  database  is  stored,  and  provides  support  far 
database  development,  system  maintenance  and  hardware  diagnostics. 


The  image  processor  in  this  simplest  configuration  consists  of  one  each 
of  four  basic  unit  types,  and  would  occupy  two  standard  equipment  racks.  The 
addition  of  system  units  to  increase  resolution  or  add  channels  is  accomplished 
as  indicated  in  the  figure.  The  system  is  structured  as  a  pipeline  of 
processing  units  where  computing  tasks  are  distributed  along  a  sequence  of 
dedicated  computing  elements.  The  cross-hatched  areas  in  Figure  1  represent 
the  several  field  buffer  memories  in  the  system.  Cne  television  field  time  is 
allocated  to  transport  and  process  data  from  one  field  buffer  to  the  next,  thus 
each  device  can  operate  at  its  own  pace  and  be  designed  to  accomodate  only  the 
average  complexity  of  sin  image  field. 


The  abject  manager  and  polygon  manager  comprise  what  is  called  the 
viewpoint  processor .  They  perform  image  computation  functions  which  sure 
system-wide  (i.e.  channel  independent ) .  These  functions  include  the  processes 
required  to  identify  the  necessary  subset  of  the  database  required  for  a 
specific  eye  position,  and  fetch  and  process  this  data  to  an  appropriately 
selected  set  of  scene  elements  Which  are  potentially  on-screen,  front-faced  and 
at  their  proper  level  of  detail. 

The  geometric  processor  performs  all  confutations  required  to  transform 
the  incoming  model-space  description  of  scene  elements  into  a  two-dimensional 
perspective  image  in  display  space.  These  steps  include  rotation,  clipping  and 


465 


perspective  division.  The  result  of  this  process  is  an  image-plane  description 
of  all  scene  data,  which  is  stored  in  the  polygon  buffer. 

The  remaining  steps  to  generate  the  final  image  are  accomplished  in  the 
display  processor,  which  performs  hidden-surface  removal  and  "scan  conversion", 
although  both  operations  proceed  in  significantly  different  ways  than 
heretofore.  The  display  processor  also  applies  the  effects  of  fog, 
illumination,  shading  and  color. 


THE  OBJECT  MANAGER 

The  object  manager  performs  three  main  functions :  management  of  the 
database  from  the  magnetic  disk  to  the  management  and  geometry  memories; 
culling  of  the  database  to  extract  the  minimal  set  of  required  scene  elements 
for  a  given  eyepoint;  and  ordering  of  the  scene  elements  into  visual-priority 
order  (front  to  back).  Its  main  parts  are  the  management  memory,  the  cell 
processor,  and  the  priority  processor. 

The  cell  processor  traces  a  database  tree,  discovering  and  outputting 
pointers  to  collections  of  scene  elements  as  branches  of  the  tree  are 
traversed.  The  local  depth  of  tree  tracing  depends  on  the  results  of  the 
various  database  management  tests,  and  the  order  of  tree  tracing  is  determined 
by  the  priority  processor,  which  orders  the  scene  elements  into  visual  priority 
order  according  to  the  procedure  specified  in  each  local  collection  of  scene 
elements.  In  order  for  this  process  to  be  better  understood,  we  pause  here  and 
discuss  the  database  structure  itself. 


Database  Organization  and  Structure 

The  database  is  a  hierarchical  tree  structure  of  geographically 
organized  nested  levels  of  detail.  Embedded  within  the  database  structure  is 
information  required  to  perform  a  variety  of  culling  tests.  The 
inter-relationship  of  the  tree  parts  provides  a  structure  Which  is 
self-managing.  The  nature  of  this  structure  will  become  more  apparent  as  we 
define  and  use  the  following  terms: 

A  scene  element  is  either  a  polygon  or  a  light.  The  modeler  can  specify 
the  reflectance  at  each  vertex  of  a  polygon,  or  he  can  have  the  image  generator 
calculate  each  vertex  reflectance  by  performing  a  sun  calculation  once  per 
polygon  (flat  shading),  or  once  per  vertex  (smooth  shading) .  The  il  leler  also 
specifies  a  color  for  each  polygon,  and  may  specify  special  features  such  as 
color  blending.  Lights  may  be  specified  individually  or  as  strings,  where  a 
string  may  be  straight  or  curved.  The  modeler  can  also  specify  directional, 
flashing,  rotating  and  strobing  characteristics  for  lights. 

An  object  is  a  collection  of  scene  elements.  The  scene  elements  within 
an  object  are  usually  related  by  structure  or  geographic  proximity,  and  the 
order-of-encounter  of  scene  elements  within  an  object  is  assumed  to  be  their 
visual  priority  order.  Objects  can  be  specified  as  transparent.  Several 
levels  of  transparency  are  provided,  giving  the  modeler  great  flexibility  in 
representing  important  cues  such  as  the  pilots  helmeted  head  inside  a  canopy. 
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A  cell  is  a  volume  in  model  space  represented  by  a  decision  node  in  the 
database  tree.  A  cell  can  have  up  to  two  option  pointers  and  a  transition 
range  which  is  used  to  choose  between  them.  Associated  with  the  cell  is 

sufficient  information  to  perform  a  coarse  field-of-view  test.  This  POV  test 
is  done  so  as  to  be  valid  for  all  database  elements  that  subsequent  portions  of 
the  database  tree  may  invoke. 

An  option  pointer  is  a  pointer  to  either  an  object  or  a  mesh.  If  the 
pointer  references  an  object,  then  the  tree  structure  along  this  path  is 

finished.  If  the  pointer  references  a  mesh,  then  more  tree  tracing  may  be 
done.  As  the  cell  processor  traces  the  tree,  object  pointers  encountered  ^re 
output  to  the  polygon  manager  while  mesh  pointers  cause  the  cell  processor  to 
reenter  its  mesh  processing  algorithm. 

A  mesh  is  a  collection  of  cells.  Included  within  the  mesh  is 

information  which  allows  the  visual -priority  ordering  of  the  cells  to  be 

accomplished.  They  can  be  f i xed-or der ed ,  range-sorted  (by  the  real-time 
software),  or  ordered  by  the  structure  of  planes  and  relationships  contained 
within  the  mesh.  This  last  approach,  while  using  some  ccranon  notions  such  as 
separating  planes  and  priority  relationships,  is  far  more  flexible  than  the 
usual  binary  tree  method.  Modeling  constraints  are  considerably  relaxed,  since 
priority  structure  is  confined  to  local  geographic  regions,  limiting  priority 
interactions  to  relatively  small  collections  of  scene  data.  In  addition, 
complex  three-dimensional  geometry  which  would  be  impossible  to  resolve  with 
binary  trees  is  easily  done  without  the  need  to  perform  artificial  subdivision 
of  polygons  and  objects  along  various  separating  planes. 

Culling  tests  are  performed  to  eliminate,  as  early  as  possible  and  in 
pieces  as  large  as  possible,  those  portions  of  the  database  which  vri.ll  not 
appear  in  ary  display  channel.  These  culling  tests  include  level -of -detail 
(only  the  simplest  representation  of  an  object  necessary  for  a  given  viewing 
distance  is  used),  field-of-view  testing  (does  the  item  impinge  on  any  display 
channel) ,  backface  test  (polygons  Whose  visible  sides  face  away  from  the 
eyepoint),  and  perspective  area  (polygons  whose  projected  screen  area  is 
smaller  than  a  systeo-oontrolled  threshold).  Level -of -detail  and  field-of-view 
tests  cure  applied  in  the  object  manager  in  a  nested  hierarchical  fashion; 
field-of-view,  backface  and  perspective-area  tests  are  done  in  the  polygon 
manager  on  a  scene-element  basis. 

The  database  tree  is  the  structure  of  meshes  and  objects  organized  by 
their  association  into  cells.  Each  item  in  the  database  is  uniquely  associated 
with  both  its  parent  (calling)  item,  and  items  it  references  (its  sons).  The 
structure  thus  provides  for  a  hierarchical  processing  of  the  database, 
controlled  by  level-of-detail  considerations,  culled  by  field-of-view 
constraints,  and  ordered  into  visual  priority. 

The  Cell  Processor 


The  cell  processor  traces  the  database  tree  once  every  field,  output ing 
the  proper  sequence  of  object  pointers  to  the  polygon  manager  to  draw  the 
image.  A  field  is  initiated  when  the  cell  processor  is  given  an  eyepoint  and 
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the  database  visual  environment  mesh  pointer.  When  a  mesh  is  processed  the 
priority  processor  determines  the  output  order  of  the  cells  while  the  cell 
processor  performs  field-of-view  and  transition-range  calculations.  Cells 
which  don't  impinge  on  any  channel  are  not  output  and  all  further  processing  on 
these  cells  ceases .  The  regaining  cells  are  output  in  their  priority  order, 
and  the  transition  range  calculations  are  used  to  determine,  for  each  cell, 
which  option  pointer  will  be  pursued.  Since  the  visual-priority  ordering  of 
cells  in  each  mesh  applies  to  the  scene  elements  which  the  cells  will 
ultimately  invoke,  when  a  mesh  pointer  cell  is  output  the  new  mesh  must  be 
processed  before  resuming  the  output  of  cells  in  the  current  mesh.  Thus  the 
algorithm  must  be  reentrant .  When  a  mesh  is  completed  (i.e.  when  all  of  its 
cells  have  been  output  and  none  invoked  new  meshes),  the  algorithm  "pops"  back 
to  the  previous  level  of  processing  to  resume  the  previous  mesh.  The 
processing  of  the  database  tree  proceeds  in  this  manner  until  all  branches 
specified  by  the  level -of-detail  tests  have  been  traced,  and  all  object 
pointers  have  been  output  to  the  polygon  manager. 

The  Pager 

Implemented  within  the  object  manager  as  a  background  task  is  an 
algorithm  whose  function  is  to  manage  the  physical  memory  so  that  potentially 
required  database  scene  elements  cure  resident.  The  pager  embodies  many  of  the 
database  management  concepts  from  previous  Evans  &  Sutherland  systems,  but 
executes  those  processes  within  the  hardware  at  greatly  increased  efficiency. 
A  consequence  of  this  is  the  ability  to  manage  databases  whose  actual  size  is 
limited  only  by  the  fixed  storage  medium  (the  disk),  and  with  speed  limited 
only  by  the  disk  seek/ transfer  rate. 

The  pager  traces  the  database  tree  in  much  the  same  fashion  as  the  cell 

processor?  however,  no  consideration  is  given  to  the  visual-priority  ordering 

of  database  elements,  and  no  field-of-view  rejection  is  performed.  A  pager 
pass  is  initiated  when  the  PDP-11  sends  the  pager  an  eye  vector  and  the 

database  input  mesh  pointer.  The  database  tree  is  processed  for 
level-of-detail  decisions,  and  the  pager  checks  that  each  item  required  is 
resident  in  physical  memory.  As  it  traces  the  tree  it  discovers  items  needed 
but  not  resident.  These  are  requested  from  the  PDP-11  and  are  sent  from  the 
disk  via  "direct-memory-access"  node.  The  pager  also  identifies  database 
blocks  which  are  resident  but  not  required,  and  such  memory  blocks  are 
"garbage-collected"  for  later  use. 

This  approach  to  managing  the  disk/memory  interface  has  several 

advantages  over  software  based  methods.  First,  the  amount  of  work  that  the 
pager  must  do  is  related  not  to  the  total  database  size,  but  only  the  portion 
of  the  database  required  for  a  specific  viewpoint.  Thus  the  pager  load  is  tied 
to  the  local  database  density,  and  order -of -magnitude  increases  in  database 
size  cause  almost  insignificant  increases  in  pager  work-load.  Second,  the 
process  of  tracing  the  tree  is  very  fast — typically  the  entire  on-line  database 
can  be  traced  and  update  requests  identified  in  much  less  than  a  second.  Thus 
the  system  responds  very  rapidly  to  changing  database  scene-element  needs,  and 
the  total  amount  of  data  required  to  be  on  hand  can  be  reduced  to  a  minimum. 
Third,  the  process  is  entirely  driven  by  the  database  structure  itself — the 
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real-time  software  needs  to  know  very  little  about  the  database  to  support  its 
management. 


THE  POLYGON  MANAGER 

The  polygon  manager  performs  sun  illumination  calculations  as  well  as  a 
number  of  additional,  increasingly  discriminating  database  culling  tests. 
Included  within  the  polygon  manager  is  the  geometry  memory,  which  contains  the 
vertex  and  structural  data  for  the  scene  elements,  as  well  as  information 
needed  for  the  various  inclusion  tests. 

Processing  in  the  polygon  manager  is  directed  by  the  information  stored 
in  the  object  buffer.  This  field  buffer  contains  pointers  to  the  various 
objects  required  for  each  channel,  as  well  as  control  and  transformation 
information.  The  nature  of  this  structure  allows  constants,  control  and 
geometry  to  be  processed  in  an  intermixed  mode,  making  it  possible  to 
accomodate  special  requirements  such  as  split  screen  and  multiple  eyepoints. 
Also,  objects  can  be  preceded  by  offset  vectors  to  allow  instancing  (multiple 
use  of  scene  details ) . 

The  polygon  manager  does  the  series  of  tests  mentioned  above  to  attempt 
to  discard  each  scene  element.  Scene  elements  which  survive  all  tests  continue 
processing  as  appropriate  to  their  type.  For  polygons,  the  shading  type  is 
determined  and  sun  illumination  calculations  are  performed  for  either  smooth, 
flat  or  fixed  shading.  Each  scene  element  is  then  output  to  the  geometric 
processor.  The  required  channel-specific  transformation  matrices  are  sent  to 
the  geometric  processor  prior  to  any  scene  elements  that  need  them. 

THE  GEOMETRIC  PROCESSOR 

The  geometric  processor  receives  polygons  in  model  space  coordinates;  it 
also  receives  eye  vectors  and  rotation  matrices  specific  to  its  channel.  The 
geometric  processor  first  transforms  the  scene  element  to  channel  coordinates, 
then  performs  clipping  to  eliminate  portions  of  the  polygon  or  light  string 
which  lie  off-screen.  Light  strings  are  expanded  and  the  directionality  of 
each  individual  light  is  calculated  according  to  specific  lobe  parameters. 
On-screen  vertices  undergo  perspective  division  and  are  scaled  to  the 
resolution  of  the  screen,  which  is  much  higher  than  the  number  of  lines  or 
elements.  The  resultant  image-plane  description  of  all  the  scene  elements  is 
stored  in  the  polygon  buffer. 


THE  DISPLAY  PROCESSOR 

The  display  processor  performs  the  set  of  tasks  required  to  transform  a 
display-plane  representation  of  the  image  into  a  composite  picture  with  hidden 
portions  removed,  image  quality  preserved,  special  effects  applied,  and  T.V. 
formatting  accomplished.  In  CT-5,  the  display  processing  function  abandons  the 
notions  of  scan-line  processing,  and  capitalizes  on  the  considerable  advantages 
of  area-processing,  feature-sequential  image  processing,  and  analytic  handling 
of  image  descriptors  to  avoid  aliasing  and  greatly  extend  performance. 
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The  display  processor  operates 
cn  rectangular  areas  of  the  screen 
called  spans .  A  span  may  be  thought 
of  as  a  collection  of  display  pixels 
which  are  computed  in  parallel . 

Individual  scene  elements  cure 
submitted  to  the  span  processor  in 
visual  priority  order,  and  the  span 
processor  calculates  the  image  for 
each  span  which  is  thus  influenced. 

Hie  time  to  compute  a  span  is  not  a 
significant  function  of  the  complexity 
of  the  scale  element  internal  to  the 
span,  thus  the  time  to  compute  a  scene 
element  is  nearly  proportional  to  its 
size.  One  can  see  how  the  principal 
processing  functions  are  accomplished 
by  referring  to  Figure  2. 

Three  memories  participate  in 
the  process .  Hie  polygon  buffer 
contains  image-plane  descriptions  oF 
all  the  scene  elements  ( such  as 
polygons  and  lights)  that  are  to 
appear  on  the  display.  The  video  Figure  2 

assembler  is  a  dual  video  field  buffer 

with  storage  for  the  red,  green  and  blue  components  at  pixel  resolution.  This 
"ping/pong"  memory  is  written  into  on  one  side  by  the  span  processor  while  the 
other  side  provides  video  output  to  the  display.  The  mask  memory  contains  a 
high-resolution  coiqposite  record  of  all  prior  image  regions  processed  in  a 
field,  but  it  includes  only  the  geometric  details  delineating  the  presence  or 
absence  of  scene  data. 

As  each  new  polygon  is  submitted  to  the  span  processor,  an  analytic 

description  of  the  polygon  within  each  touched  span  is  prepared.  The  mask 

memory  is  used  to  "erase"  portions  of  this  analytic  description  which 
correspond  to  portions  of  the  span  Which  are  already  occulted.  The  remainder, 
which  denotes  the  visible  part  of  the  current  polygon,  is  convolved  with  the 
spatial  filter  to  derive  the  display  video  contribution  of  this  polygon  within 
the  span.  The  incremental  effects  of  the  polygon  are  summed  into  the  video 
assembler  memories  after  accounting  for  visibility  effects,  smooth  shading, 
color  blending,  transparency,  and  color.  The  analytic  description  of  the 
geometry  is  then  added  to  the  ma3k  memory  to  occult  future  submissions  to  this 
span. 

Severed,  important  concepts  embodied  in  this  process  account  for  the 
exceptional  image  quality  thus  achieved.  First,  since  the  image  presented  to 
the  spatial  filter  is  a  faithful,  high  resolution  rendition  of  the  actual 

geometry  involved,  errors  due  to  quantization  of  the  image  to  scan-line 

resolution  are  avoided.  Second,  since  image  processing  proceeds  on  the  basis 
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of  rectangular  areas  on  the  screen,  all  imagery  is  treated  identically,  without 
complexity  limits,  and  without  directional  biases  related  to  raster 
orientation.  Third,  cpatial  filtering,  which  is  performed  at  the  pixel  level 
with  a  filter  approximately  two  pixels  wide,  attennuates  the  higher  spatial 
frequencies  smoothly  and  without  loss  of  phase  information.  These  image 
processing  techniques  produce  images  with  high  subjective  resolution,  allowing 
useful  presentation  of  scene  details  with  dimansions  much  smaller  than  the 
pixel  size. 

Capacity  characteristics 

The  area-based  nature  of  the  display  processing  algorthm  gives  rise  to 
an  interesting  capacity-versus-time  relationship.  We  can  gain  an  intuitive 
insight  into  the  nature  of  this  relationship  by  imagining  a  basic  image  Which 
we  wish  to  make  more  complex,  and  by  observing  the  way  in  Which  the  display 
processor  responds. 

we  can  increase  the  richness  of  an  image  in  two  basic  ways.  One  is  to 
increase  the  detail  of  an  existing  object,  in  this  case,  the  image  area  of  the 
more  detailed  object  will  be  about  the  same  as  the  simpler  one.  Although  the 
span  processing  is  based  on  area,  seme  additional  time  is  required  due  to  the 
fact  that,  as  scene  elments  become  smaller,  the  efficiency  with  which  they  fit 
the  spans  decreases.  In  this  situation,  then,  the  processing  time  grows 
somewhat  as  surfaces  are  added,  but  much  less  than  linearly. 

The  second  way  we  can  increase  scene  richness  is  to  add  more  objects. 
In  this  case,  the  depth  complexity  of  the  scale  increases  (i.e.  the  image  has, 
an  an  average,  more  layers).  However,  since  we  are  building  the  image  up  from 
nearest  to  farthest  surfaces,  we  can  skip  over  portions  of  the  image  which  have 
previously  been  totally  occulted.  Such  a  "full-span"  mechanism  can  help  to 
bring  the  net  processing  time  for  a  frame  significantly  closer  to  that  for  a 
tiled  image  which  has  no  occulted  parts. 

The  net  effect  of  these  characteristics 
is  to  give  rise  to  a  capacity  versus 
processing-time  relationship  with  the  general 
characteristic  shewn  in  Figure  3.  The 
interesting  property  of  this  timing  function  is 
that  its  slope  is  appreciably  steeper  than  one; 
i.e.  that  small  percentage  increases  in 
processing  time  allow  large  percentage 
increases  in  visual  complexity.  This 
characteristic  nakes  it  very  attractive  to 
trade  field  time  for  image  complexity  when 
needed. 


Figure  3 


USING  LEVELr-CF-OETAIL 

Figure  4  shows  one  potential  use  of  multiple  levels-of-detail .  Although 
five  levels  were  used  in  the  example,  the  system  can  effectively  support  an 
unlimited  number  of  levels  of  detail.  The  modeler  provides  the  number  of 
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levels  needed  for  each  specific  application  to  optimize  tradeoffs  between  scene 
fidelity  and  load  requirements.  He  specifies  the  transition  ranges  based  on 
the  various  compromises  that  apply  to  the  geometry,  color  and  contrast  of 
adjacent  levels  of  detail,  focusing  the  system  resources  where  they  are  most 
needed. 


The  pager  builds  the  database  tree  into  memory  beginning  with  the  lowest 
levels  of  detail  (i.e.  the  simplest  representations),  and  it  retains  in  memory 
all  levels  up  through  the  ones  currently  needed.  As  is  evident  in  the  example, 
the  relative  cost  of  keeping  around  all  simpler  versions  of  something  is  about 
equal  to  the  cost  of  the  most  complex  version  stored.  This  cost  in  memory 
space  is  a  bargain,  since  it  allows  the  system  to  fall  back  instantaneously  to 
less  complex  representations  of  scene  elements  that  are  already  in  memory.  The 
practical  advantage  is  that  there  is  effectively  no  restriction  on  how  fast  the 
system  can  respond  to  load  management  needs,  while  traffic  at  the  disk 
interface  is  minimized.  Note  that  levels  of  detail  more  complex  than  currently 
required  are  not  brought  into  memory,  so  only  the  memory  needed  to  actively 
support  the  current  image  (and  its  fallback  lower  levels  of  detail)  is  in  use. 
Thus  the  system  effectively  uses  the  available  memory,  closely  relating  memory 
usage  with  the  displayed  scene  complexity  rather  than  the  total  database  size. 

The  extensive  use  of  level-of -detail,  and  the  rapid  response  time  of  the 
system  to  changing  database  scene  element  needs,  provide  the  capability  to 
manage  very  large  databases  while  providing  high  local  scene  complexity.  These 
characteristics  are  inqportant  prerequisites  to  effectively  performing  sane 
simulation  tasks  which  are  of  particular  importance  in  military  applications. 
Per  instance,  high-speed  low-level  flight  over  mountainous  terrain  or  heavy 
forest  can  be  effectively  supported  by  CT-5  while  maintaining  high  scene 
oanplexity  and  image  quality. 


RESULTS 

The  architectural  concepts  described  above  lead  to  seme  interesting  and 
useful  system  properties.  The  following  sections  summarize  the  resulting 
system  characteristics  that  may  be  of  particular  interest  to  those  Who  plan, 
specify  and  apply  CIG  systems. 

Overload  Response 

Overload  in  CIG  visual  systems  is  inevitable.  Regardless  of  the 
specified  capacity  of  a  system,  the  user  will,  in  time,  exceed  it.  Thus  the 
manner  in  which  a  system  responds  to  overload  is  very  important  to  the  user. 
Two  systems  with  the  same  nominal  capacity  can  prove  to  be  quite  different  in 
practice  when  compared  on  the  basis  of  effective  performance.  Sane  systems  are 
characterized  by  having  absolute  capacity  limits  which  cure  at  or  near  their 
nominal  performance  limits.  Breaching  these  limits,  even  by  a  small  amount, 
can  result  in  drastic  image  errors.  The  cnly  available  strategy  in  such  cases 
is  to  tune  the  model  so  that  overload  does  not  occur — even  briefly.  This 
tuning  is  accomplished  typically  by  limiting  model  density  and  by  carefully 
structuring  the  level  of  detail  transitions  to  prevent  overload  -  a  costly 
undertaking  that  generally  results  in  under-utilization  of  the  system. 


gure  4C:  69  Polygons 


CT-5  systems  automatically  manipulate  image  generator  resources  to 
produce  ccntrolled  responses  over  a  broad  range  of  impending  and  actual 
overload  conditions.  Thus,  the  system  may  be  operated  at  much  higher  average 
load  levels,  and  model  tuning  requirements  can  be  relaxed  or  adjusted  to  suit 
the  training  mission  needs.  This  fortunate  impact  on  modeling  requirements 
will  become  increasingly  important  with  the  advent  of  larger  and  larger  data 
bases  as  automated  modeling  techniques  are  employed  and  hand  tuning  becomes 
prohibitive. 

Three  principal  mechanisms  cure  employed  to  accomplish  load  management 
objectives: 

1.  CT-5  capacity  increases  with  available  computing  time.  This 
characteristic,  previously  described  for  the  display  processing 
functions,  also  holds  for  the  remainder  of  the  system,  where  the  only 
limits  beside  computing  time  are  certain  memories,  and  they  are 
generously  sized  relative  to  nominal  requirements. 

2.  Polygons  are  rejected  based  on  their  perspective  size.  This  size 
threshold  can  be  adjusted  to  provide  continuous  control  over  the  number 
of  polygons  presented  to  the  pipeline. 

3 .  Overall  scene  loading  can  be  ccntrolled  by  level-of -detail 
selections  and  the  range  thresholds  at  which  switching  occurs.  These 
thresholds  can  be  changed  dynamically. 

The  general  purpose  computer  monitors  load  at  various  points  in  the 
image  processor  and  also  controls  the  field  time,  under  normal  conditions  the 
field  time  is  kept  constant.  Intending  overload  conditions  are  sensed  and 
regulated  by  changing  the  thresholds  on  perspective  size  and  level  of  detail. 

If  scaie  conditions  should  change  so  rapidly  that  units  cannot  complete 
their  tasks  in  the  allotted  field  time,  then  the  field  time  is  simply  extended 
to  provide  immediate  relief  for  the  problem.  Thus  the  instantaneous  penalty 
far  overload  is  a  lower  field  rate. 

Field  rate  reduction  is  viewed  as  a  temporary  measure  to  be  used  while 
more  gradual  adjustments  sure  made  on  level  of  detail  and  perspective  size 
parameters.  These  changes  reduce  the  load  by  rejecting  increasingly  larger 
polygons.  Thus  the  long  term  penalty  is  that  potentially  perceptible  polygons 
are  rejected  and  alternate  scene  definitions  come  into  use  sooner. 

At  intermediate  values  of  these  thresholds  it  becomes  preferable  to 
accept  operation  at  a  lower  field  rate  rather  than  to  reject  scene  details  of 
obvious  and  useful  size.  The  principal  effect  of  a  lower  field  rate  is  to 
introduce  the  possiblity  of  perceptible  flicker.  Field  time  extension  is 
therefore  limited  to  preclude  excessive  flicker. 

Under  still  more  severe  conditions  the  strategy  resorts  to  frame  rate 
update.  This  doubles  the  processing  time  (and  capacity)  for  all  units  up  to 
the  display  processor.  As  a  final  measure,  the  display  processor  can  resort  to 
a  frame  rate  update  cycle  with  field  rate  display. 
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In  summary,  the  overload  response  of  CT-5  is  designed  to  fit  the  penalty 
to  the  crime.  The  result:  graceful  overload  response  and  more  useful  imagery 
on  the  screen  more  of  the  time. 


Cne  of  the  principal  objectives  for  CT-5  was  to  improve  image  quality 
well  beyond  that  available  from  the  best  existing  CIG  systems.  The  CT-5  system 
approach  implements  a  fundamentally  sound  and  comprehensive  solution  to 
aliasing  problems.  In  the  course  of  developing  and  evaluating  these  concepts 
we  designed  a  test  pattern  that  was  found  to  be  quite  discriminating  in  issues 
of  image  quality.  A  photograph  of  the  pattern  as  generated  by  a  CT-5  system  on 
a  700  line  odor  display  is  shown  in  Figure  5. 

The  pattern  consists  of  triangles  radiating  from  a  common  center.  The 
triangles  are  defined  as  maximum  intensity  polygons  against  a  blade 
background.  At  the  periphery  of  the  pattern  each  triangle  is  one  pixel  wide 
and  the  space  between  neighboring  triangles  is  four  pixels.  The  pattern 
reveals  much  about  system  performance. 

The  first  thing  to  note  from  the  photograph  is  that  there  is  very  little 
variation  in  the  appearance  of  the  triangular  spokes  as  a  function  of  their 
orientation.  The  system  tends  to  treat  scene  details  equally,  without 
directional  biases. 


477 


Figure  6 


The  individual  spokes  are  resolvable  towards  the  center  well  beyond  the 
hell f -way  point.  Thus  we  see  that  the  system  is  able  to  consistently  resolve 
detail  that  measures  less  than  one  half  pixel  in  size,  and  the  detail  appears 
in  proper  positional  relationship  (phase)  with  similar  neighboring  elements. 
This  aspect  of  the  pattern  indicates  that  hich  effective  resolution  is  achieved 
and  that  small,  generally  isolated  segments  of  scene  detail  can  be  visually 
tracked  down  to  fractional  pixel  size. 

The  final  observation  about  the  test  pattern  performance  is  that  the 
spokes  blend  uniformly  into  the  central  region.  Hie  central  part  of  the 
pattern  maintains  a  generally  constant  character  and  average  intensity. 
Pattern  modulation  decreases  uniformly  and  no  coherent  patterns  arise  toward 
the  centred,  area  as  would  be  characteristic  of  aliased  spectra.  This  property 
allows  small  objects  to  be  portrayed  and  recede  unobtrusively  into  their 
background. 


Figure  6  shows  all  this  theory  being  applied  to  a  typical  modeling 
situation,  and  illustrates  the  detail  and  quality  of  images  drawn  by  the  CT-5 
display  processor.  Note  particularly  the  runway  cracks,  which  are  modeled  .05 
feet  wide  (about  5/8  of  an  inch)  on  30  foot  grids.  The  cracks  toward  the  ri^it 
side  of  the  runway  image  are  still  clearly  evident  and  well  behaved,  even 
though  they  cure  only  about  1/7  of  a  pixel  wide. 


The  basic  CT-5  system  can  be  expanded  both  in  display  resolution  and  in 
number  of  channels  by  simply  adding  more  units  of  the  types  already  described. 
Part  of  the  overall  system  strategy  was  to  provide  an  approach  to  expansion 
which  did  not  cause  degradation  of  individual  channel  capacity  as  resolution  car 
number  of  channels  was  increased.  The  modularity  of  CT-5  building-block 
components  allows  easy  and  direct  expansion  with  negligible  impact  on 
per -channel  performance. 

Channel  Expansion.  Channel  expansion  in  CT-5  is  achieved  by  the 
addition  of  channel-specific  parallel  hardware  at  an  early  stage  in  the 
processing  pipeline.  These  modular  system  increments  provide  independent 
computing  capacity  for  each  channel,  thereby  decoupling  the  performance  of  any 
one  channel  from  that  of  the  other  channels.  This  also  makes  it  possible  to 
conduct  simultaneous  independent  simulations  within  one  CT-5  visual  system 
without  the  danger  of  unpredictable  interactions  among  the  simulations.  CT-5 
is  designed  to  acocmodate  a  variety  of  other  special  requirements  of  this 
type.  Independent  channels  also  greatly  simplify  fault  isolation  and  make  it 
hi^ily  probable  that  failure  effects  will  allow  training  to  continue  on  the 
remaining  channels. 

The  parallel  channel  approach  used  in  CT-5  contrasts  with  the 
organization  commonly  employed  by  most  other  CIG  systems,  wherein  a  single 
image  processing  resource  is  shared  among  all  channels.  That  architecture 
minimizes  the  channel  specific  hardware,  but  the  available  system  capacity  must 
be  divided  in  direct  proportion  to  the  number  of  channels  served.  Thus,  both 
model  and  scene  densities  must  be  reduced  as  channels  are  added.  Furthermore, 
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secondary  system  constraints  such  as  edge  crossings  per  scanline  or  total 
segments  often  remain  constant  with  increasing  channels,  and  become  serious 
limits  when  applied  to  the  composite  of  all  channels. 

Increased  Resolution.  Increased  raster  resolution  is  accomplished  by 
the  modular  addition  of  display  processing  units.  The  single  DP  unit  used  in 
the  basic  configuration  provides  a  raster  of  1/4  million  pixels  -  equivalent  to 
a  525  line  T.V.  standard.  Expansion  to  700  line  resolution  has  been 
accomplished,  and  provision  has  been  made  to  accomodate  1000  line 
requirements.  The  relative  distribution  of  available  pixels  between  lines  and 
elements  is  quite  flexible,  thus  permitting  a  wide  variation  in  the  aspect 
ratio  of  the  chosen  display.  CP  units  are  designed  to  operate  in  parallel 
configurations  that  divide  the  span  processing  load  so  that  there  is  no 
attendant  sacrifice  in  channel  capacity  as  resolution  is  increased . 

CCNCLUSICN 

A  system  strategy  has  been  presented  which  addresses  the  problem  of 
optimizing  CI6  image  content.  This  strategy  relies  on  carefully  considering 
the  interrelationship  of  all  the  parts  that  make  up  a  computer  image  generator, 
from  the  database  to  the  display.  It  includes  some  novel  concepts  that  offer 
effective  solutions  to  perennial  problems  encountered  in  real-time  visual 
systems.  CT-5  applies  these  concepts  in  an  integrated  approach  that 
capitalizes  on  an  increased  understanding  of  the  intimate  relationship  between 
the  image  generator  and  the  data  structures  that  drive  it. 

A  hierarchical  nested  database  architecture  provides  the  driving 
structure  far  a  novel  approach  to  database  management.  This  approach  not  only 
frees  the  modeler  from  many  previously  troublesome  constraints,  but  allows  the 
image  generation  hardware  to  make  optimum  use  of  its  resources  in  providing 
training  imagery  of  maximum  complexity.  Special  attention  is  given  to  the 
problem  of  managing  extremely  large  databases,  while  insuring  that  images  of 
maxinuo  detail  are  continuously  displayed,  and  performance  limitations  imposed 
by  software  management  algorithms  are  removed.  The  hardware  implementation  of 
some  very  powerful  database  management  concepts  enables  CT-5  to  satisfy 
simulation  needs  which  heretofore  have  gone  unmet,  expanding  the  sphere  of 
effectiveness  of  the  simulation  excercise. 

The  CT-5  display  processor  makes  a  substantial  departure  from  the 
scan-line  approaches  currently  used,  and  sets  new  standards  of  image  quality 
and  capacity.  The  rigorous  approach  to  the  preservation  of  image  quality,  and 
the  avoidance  of  aliasing  by  analytic  methods  allows  CT-5  to  provide  imagery  of 
unexcelled  quality  and  resolution,  while  the  parallel  area  processing  of  scene 
features  eliminates  ccnplexity  limits  imposed  by  scan-line  based  systems. 

Hie  architectural  properties  which  govern  the  performance  parameters  of 
the  system  relate  processing  time,  degree  of  area  parallelism  and  capacity  in 
an  intrigueing  way.  This  behavior  has  made  possible  the  provision  of  an 
extremely  elastic  overload  response  within  the  context  of  high  basic  capacity. 
The  power  of  the  integrated  CT-5  approach  to  load  management  suggests  new 
directions  in  database  design.  In  the  future,  perhaps  the  objective  will  be  to 
provide  appreciably  more  model  detail  them  usual,  while  allowing  the  load 


management  capability  of  the  system  to  keep  the  displayed  complexity  of  the 
visual  environment  at  the  nominal  system  performance  limits.  As  database 
generation  processes  become  more  and  more  automated,  and  as  suppliers  and  users 
of  C1G  visual  systems  gain  increased  insight  into  the  complex  problem  of 
supplying  the  iiqportant  training  cues  relevant  to  each  mission,  the  value  of 
this  approach  will  increase. 
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Area  of  Interest  -  Instantaneous 
Field  of  View  Vision  Model 


ABSTRACT 

Experiments  have  been  carried  out  using  a  limbus  eye  tracker  and  a  static 
projected  scene  with  a  servo  driven  high  resolution  area.  The  results  of 
these  experiments  are  being  used  to  define  vision  model  specifications  for  a 
Helmet  Mounted  Display  under  development  at  the  Naval  Training  Equipment 
Center. 

INTRODUCTION 

The  ultimate  goal  of  the  Helmet  Mounted  Display  (HMD)  is  to  provide  a 
pilot  with  a  displayed  scene  which  is  indistinguishable  from  the  real  world  in 
terms  of  pilot  performance  on  the  required  task.  Vision  model  tests  have  been 
conducted  at  NTEC  which  led  in  part  to  the  specifications  of  an  optimum  HMD. 
(See  Figure  1).  The  optimum  HMD  specifications  also  included  trade-off 
considerations  among  all  the  system  parameters.  The  HMD  concept  is  discussed 
in  detail  in  another  paper  in  these  proceedings. 

Present  visual  simulation  technology  does  not  provide  for  all  the 
necessary  variables  to  simulate  the  real  world  completely.  Therefore,  the 
actual  display  obtained  must  be  a  trade-off  among  the  available  parameters. 

One  important  parameter  is  the  physical  limitation  of  the  human  in  the  loop. 
For  example,  it  is  not  necessary  to  provide  a  high  detail  scene  in  a  non-look 
direction.  In  fact,  psycho-physical  experiments  are  indicating  that  realism 
can  be  obtained  by  providing  a  high  resolution  Area  of  Interest  (AOI)  insert 
within  a  larger  Instantaneous  Field  of  View  (IFOV)  to  match  the  resolution 
capabilities  of  the  human  eye.  The  visual  acuity  of  the  human  eye  is  one 
minute  of  arc  in  the  fovea  of  the  retina.  Outside  the  fovea,  the  acuity  drops 
rapidly  reaching  approximately  1/10  of  the  foveal  acuity  at  20°  from  the 
fovea.  Also,  indications  are  that  the  IFOV  does  not  need  to  cover  the  Total 
Field  of  View  (TFOV)  available  to  the  observer.  The  intention  of  the  HMD 
system  is  to  apply  these  concepts  to  a  Computer  Image  Generator  (CIG)  which  is 
limited  hy  the  throughput  delay  and  number  of  edges  which  can  be  processed  in 
real  time.  The  intent  is  to  use  two  channels  of  CIG  for  the  HMD.  One  channel 
will  be  used  for  the  small  field  AOI  and  thus  have  high  edge  density.  The 
second  channel  will  be  used  for  the  larger  field  IFOV  and  thus  have  lower  edge 
density. 

The  tests  conducted  at  NTEC  considered  the  following  as  variables:  size 
of  AOI,  size  of  IFOV,  resolution  drop  from  AOI  to  IFOV,  edge  matching  between 
boundaries,  brightness  fall  off,  eye  tracker  AOI,  eye  tracked  AOI  and  IFOV, 
and  throughput  delay  variation.  The  boundary  conditions  for  these  variables 
are  defined  by  considering  human  visual  characteristics,  input  from  our  own 
and  other  laboratories,  and  HMD-CIG  requirements.  Head  tracking  experiments 
have  not  been  completed  as  yet. 

EXPERIMENTAL  PROCEDURES 

The  system  that  was  used  for  vision  model  tests  is  a  modification  of  the 
SURNOT  (Surface  Navigation  and  Orientation  Trainer).  SURNOT  is  a  wide  angle 
display  system  designed  and  assembled  at  NTEC.  It  consists  of  a  Xenon  arc 
light  source,  a  unique  360°  annular  projection  lens,  and  a  12.5'  radius 
spherical  screen.  Figure  2  shows  an  example  of  the  imagery  produced  by  this 
annular  projection  lens.  Figure  3  shows  the  format  of  the  annular  image 
projected  through  the  lens  system. 


482 


Figure  2. 

Annular  Projection  Lens  Imagery 


the  Lens 


The  system  has  been  modified  by  placing  a  servo  driven,  variable 
resolution  mask  near  the  image  plane  as  shown  in  Figure  4.  The  mask  is  made 
of  acrylic  and  is  designed  to  reduce  the  resolution  away  from  the  line  of 
sight.  The  servo,  which  rotates  the  mask  horizontally,  can  be  controlled  by 
the  output  from  the  eye  tracker  which  is  a  Biometrics  Model  200.  The  servo 
system  was  designed  and  built  according  to  specifications  for  use  with  the  eye 
track  system.  The  servo  responds  to  an  input  of  55  mV /deg  and  has  the 
following  limitations:  position  error  <  .2°;  maximum  angular  velocity 
2  .6°/msec;  dynamic  jitter  <  5  percent;  and  throughput  delay  £  12  msec.  The 
servo  can  rotate  >  *  90*. 

A  block  diagram  of  the  entire  Vision  Model  Test  System  is  shown  in  Figure 
5;  Figure  6  shows  a  photograph  of  the  entire  test  system.  For  each  test,  a 
subject  was  positioned  in  a  chair  facing  the  screen,  head  resting  in  the 
restraint.  The  glasses  used  were  calibrated  for  the  individual.  Eye 
movements  were  sensed  by  the  eye  tracker  which  has  a  pair  of  silicon 
photo-transitors  operating  in  conjunction  with  a  gallium  arsenide  IR  source 
mounted  in  front  of  each  eye.  The  dominant  eye  was  tracked  based  on  recent 
information  from  the  literature  which  indicates  that  the  dominant  eye  can  be 
as  much  as  100  to  500  msec  behind  the  non-dominant  eye  for  certain  types  of 
eye  movement  (example:  vergences).  Eye  dominance  may  not  be  a  factor  in 
these  tests,  but  since  only  one  eye  can  be  tracked  horizontally,  the  decision 
was  made  to  track  the  dominant  eye.  The  test  used  for  eye  dominance  is  the 
”hole-in-the-card-test"  which  is  as  follows:  a  hole,  two  to  three  inches  in 
diameter  is  cut  in  an  ordinary  piece  of  cardboard  and  the  subject  asked  to 
sight  an  object  at  about  four  meters  distance  through  the  hole  (both  eyes 
open).  Closing  one  eye  and  then  the  other  should  make  it  obvious  by  observing 
parallax  effects,  which  eye  is  dominant  for  that  subject.  The  eye  tracker  was 
then  adjusted  to  track  the  dominant  eye  horizontally.  The  operating  principle 
of  the  eye  movement  monitor  depends  on  detecting  the  changes  in  reflected 
light  between  the  white  sclera  and  the  left  and  right  sides  ofothe  iris.  The 
eye  tracker  can  track  ±  20°  horizontally  to  an  accuracy  of  1/4°  depending  on 
calibration. 

The  output  from  the  eye  tracker  was  fed  into  the  electronic  delay  circuit 
which  can  vary  the  delay  from  17  to  125  msec.  This  signal  was  then 
transmitted  to  the  servo  which  rotated  the  mask  the  required  amount  to  keep 
the  high  resolution  area  along  the  line  of  sight. 

The  image  generation  system  to  be  used  with  the  HMD  has  a  given  delay 
time  before  a  new  scene  is  presented  to  the  observer.  Throughput  delay 
measurements  were  made  because  an  indication  of  the  objectionability  of  this 
delay  time  is  highly  desirable.  It  may  be  necessary  to  compensate  for  this 
delay  in  the  final  system.  Parallel  studies  are  underway  dealing  with  methods 
of  compensating  for  the  CI6  delay. 

Twenty  one  masks  were  used  in  these  tests.  The  characteristics  for  the 
various  masks  are  shown  in  Table  1.  Fourteen  subjects  were  used,  two  of  whom 
were  pilots. 

Resolution  profiles  were  obtained  for  each  mask  by  projecting  an  annular, 
high-contrast  bar  chart  and  reading  the  limiting  resolution  at  one  point  on 
the  screen  while  the  mask  was  rotated.  Figure  7  shows  the  resolution  profile 
for  mask  2C.  The  luminance  profile  was  also  obtained  for  each  mask  by  using  a 
Spectra  Pritchard  Photometer  set  up  at  eye  viewing  position  and  rotating  the 
mask.  Figure  8  shows  the  luminance  profile  for  mask  2C.  This  luminance 
profile  shows  the  extreme  of  brightness  falloff  which  the  eye  finds 
acceptable,  but  it  is  not  recommended  for  the  final  systems. 

Judgements  were  obtained  by  asking  each  subject  to  rate  the  display  on  a 
scale  of  1-4  with  the  following  rating  designations:  1  *  ideal;  2  » 
noticeable  degradation;  3  -  objectionable  degradation;  and  4  -  degradation 
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TABLE  1.  MASKS  USED 


MASK  Number 

SIZE  AOI 

SIZE  IFOV 

SURROUND 

la 

15* 

70* 

Black 

lb 

15* 

90* 

M 

lc 

15* 

110* 

M 

le 

15* 

180* 

N 

2a 

20* 

70* 

II 

2b 

20* 

90* 

II 

2c 

20* 

110* 

M 

2d 

20* 

130* 

II 

2e 

20* 

180* 

II 

3a 

30* 

70* 

II 

3b 

30* 

II 

3c 

30* 

110* 

II 

3d 

30* 

13C‘ 

II 

3e 

30* 

180* 

II 

4 

40* 

Same  ss  AOI 

II 

5 

90* 

II 

•1 

6 

90* 

II 

Grey 

7 

110* 

II 

Black 

8 

20* 

90** 

Grey 

9 

20* 

90** 

Black 

10 

20* 

no** 

II 

*  Eye  tracked  with  AOI 


expected  to  impair  operator  performance.  At  the  start  of  each  test,  the 
subject  observed  the  display  without  the  mask  but  with  all  other  test 
conditions  in  use,  i.e.,  glasses  on,  etc.  This  was  done  to  give  the  subject  a 
baseline  for  judging  the  degradation  in  the  display  as  various  masks  were 
used.  The  subject  was  not  told  the  characteristics  of  the  mask  or  the  delays 
being  used  while  tests  were  in  progress. 

A  subjective  analysis  of  discriminable  kinds  of  degradation  phenomena  was 
also  made.  Degradation  phenomena  were  described  by  the  phrases:  edges  AOI 
visible;  drop  in  resolution  visible;  edges  IFOV  visible;  granularity  of  mask 
in  low  resolution  area  visible;  and  brightness  fall-off  visible.  Additional 
subjective  comments  were  solicited  from  subjects  and  recorded. 

CONCLUSIONS 

In  analyzing  the  parameters  used  and  the  results  of  these  experiments,  it 
must  be  kept  in  mind  that  these  tests  were  designed  and  conducted  for  a  very 
specific  purpose,  i.e.,  to  aid  in  defining  acceptable  input  specifications  for 
the  Helmet  Mounted  Display  being  developed  at  NTEC.  The  tests  should  not  be 
viewed  as  comprehensive  vision  model  tests.  In  order  to  reduce  the  range  of 
vision  model  parameters  requiring  testing,  HMD  system  constraints  were  used 
based  on  simultaneous  effort  on,  for  example,  design  of  an  acceptable 
projection  lens.  In  addition,  the  following  test  equipment  limitations  are 
noted:  (a)  tests  were  conducted  with  a  high  detail  static  scene;  (b)  only 
horizontal  servoing  of  AOI,  IFOV  was  possible;  (c)  real  imagery,  not  CIG 
imagery,  was  used  (additional  tests  are  required  to  test  CIG  images,  i.e., 
"popping"  in  of  CIG  edges  as  the  level  of  detail  changes  needs  to  be 
addressed);  and  (d)  conclusions  here  are  based  on  eye  tracking  -  resource 
limitations  have  not  allowed  for  head  tracked  experiments  to  be  completed  as 
yet. 

The  averaged  ratings  obtained  from  test  subjects  for  two  groups  of  masks 
are  given  in  Figure  9  and  Figure  10.  Figure  9  shows  the  results  for  IFOV  held 
constant  and  stationary  at  130°  with  the  AOI  varying  to  15°,  20°,  and  30°.  As 
can  be  seen  from  the  graphs,  the  larger  AOI  resulted  in  more  favorable  overall 
responses.  Figure  10  shows  the  results  for  masks  where  AOI  was  held  constant 
at  30°  while  the  IFOV  varied  from  70°  to  130°.  Statistical  analysis  indicates 
that  there  is  a  significant  difference  between  the  graphs  shown.  To 
illustrate  that  the  range  of  ratings  and  the  masks  are  compatable.  Figure  11 
shows  the  range  of  response  for  various  masks  from  an  individual. 

Subject  comments  were  also  recorded.  Some  of  the  more  pertinent  comments 
related  to  various  masks  are: 

Masks  4,  5,  6  and  7:  The  motion  of  a  black,  sharp  edge  tracked  by  eye 
movement  was  very  annoying  and  apparent,  more  so  with  smaller  IFOV,  but  still 
noticeable  with  larger  IFOV.  Masks  4,  5,  and  7  went  from  a  four  rating  to 
approximately  2.5.  When  the  area  surrounding  the  IFOV  was  grey,  the  motion 
was  less  annoying  (mask  6). 

Masks  la,  lb  and  lc:  AOI  not  bad  except  as  delay  increases.  This 
results  from  the  requirement  to  track  the  eye  more  closely  when  AOI  is 
smaller.  Because  of  CIG  delay  (~  80  msec),  this  AOI  (15°)  is  most  likely  not 
acceptable  unless  some  method  is  used  to  compensate  for  this  delay. 

Masks  2a.  2b,  2c,  2d  and  2e:  70°  and  90°  IFOV  could  be  seen  by  all 
subjects.  With  small  delays  AOI  was  somewhat  noticeable,  but  not 
objectionable  to  most  observers.  One  subject  commented,  "Judgements  apply 
only  to  non-task  loaded  situation.  A  minor  distraction,  such  as  short  lag, 
might  be  noticeable  when  I  have  nothing  better  to  do  than  to  look  for  it,  but 
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might  go  unnoticed  if  I  were  involved  in  a  task  ..."  A  second  subject 
commented,  "Good  for  a  simulator  display-looks  as  good  as  a  camera  model  board 
system  (up  to  70  msec  delay).  Resolution  drop  was  no  problem,  but  brightness 
drop  and  mask  granularity  were  noticeable  at  longer  delays."  A  third  subject 
commented,  "The  picture  is  so  detailed  that  its  easier  to  pick  up  a  drop  in 
detail.  Eye  movements  are  so  fast  that  "motion"  can  be  picked  up  when  mask  is 
eye  tracked."  A  fourth  subject  commented,  "Brightness  drop  and  granularity  of 
mask  were  more  of  a  problem  than  resolution  drop."  It  should  be  noted  that 
granularity  in  the  low  resolution  area  was  a  function  that  the  mask  used  to 
drop  the  resolution  off  axis,  and  will  not  be  present  in  the  final  system. 

Masks  3a,  3b,  3c,  3d  and  3e:  70°  and  90°  IFOV  could  be  seen  by  all 
subjects.  Even  with  maximum  delay,  the  degradations  were  noticeable  but  not 
objectionable  to  subjects. 

Mask  3d:  One  subject  commented,  "No  problem  using."  A  second  subject 
commented,  "I'm  impressed."  A  third  subject  commented,  "I  can  see  the 
degradation,  but  it  wouldn't  be  a  problem  for  most  tasks  in  a  simulator." 

In  conclusion,  based  on  these  vision  model  tests  and  other  system  design 
constraints,  the  following  performance  parameters  were  chosen  for  the  HMD 
system:  AOI  =  25°  with  soft,  blended  edges;  IFOV  >_  120°  with  soft,  blended 
edges;  acceptable  resolution  drop,  AOI/IFOV  2  5/1;  acceptable  luminance  drop, 
AOI /IFOV  2  2.9/1.  These  parameters  are  compatible  with  other  system 
constraints  and  are  being  incorporated  into  the  Helmet  Mounted  Display 
System. 
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